Skip to content

Latest commit

 

History

History
1379 lines (1047 loc) · 52.9 KB

File metadata and controls

1379 lines (1047 loc) · 52.9 KB

JAPL Knowledge Base

JAPL -- Just Another Programming Language "Pure by default, concurrent by design, resource-safe by construction, distributed without apology."

Generated: 2026-03-26


Table of Contents

  1. Project Overview
  2. Core Philosophy -- 7 Principles
  3. Language Design Areas (20 Sections)
  4. Type System Specification
  5. Effect System Specification
  6. Process / Concurrency Model
  7. Memory Model
  8. Research Program -- 7 Papers
  9. Cross-Reference Map: YonedaAI Math to JAPL PL Concepts
  10. Key Design Decisions and Trade-offs
  11. Comparison Matrix

Project Overview

JAPL is a strict, typed, effect-aware functional programming language. It combines ideas from four lineages into a single coherent design:

Lineage What JAPL takes
Rust Ownership, linear types, resource safety, zero-cost abstractions
Go Simplicity, fast compilation, static binaries, built-in tooling
Erlang/OTP Lightweight processes, supervision trees, fault tolerance, distribution
FP tradition (Haskell, OCaml, Elm) Immutable values, algebraic data types, pattern matching, effect tracking, composition

The default semantics are functional: values, transformations, and composition. Mutation is available but local and explicit. Concurrency is process-based, never shared-memory-first. The runtime is simple and predictable.

One sentence: "Pure functions handle logic, supervised processes handle time and failure, and ownership handles resources."

Semantic Core

  • Immutable algebraic data types
  • First-class functions
  • Pattern matching
  • Effect tracking
  • Ownership/capability-based resources
  • Lightweight supervised processes
  • Message passing
  • Explicit error and recovery semantics

Core Philosophy -- 7 Principles

Principle 1: Values Are Primary

Data in JAPL is immutable by default. Every binding introduces a value, not a mutable variable. Values can be shared freely across processes because they cannot change. This eliminates data races at the language level and makes reasoning about program behavior local.

-- Values are the default. No `let mut`, no `var`.
let name = "JAPL"
let point = { x = 3.0, y = 4.0 }
let items = [1, 2, 3, 4, 5]

-- Transformation produces new values, never mutates in place.
let shifted = { point | x = point.x + 1.0 }
let doubled = List.map items (fn x -> x * 2)

Rationale: Immutable values compose. They can be shared across processes without synchronization, serialized without copying semantics ambiguity, and cached without invalidation logic. The Yoneda perspective reinforces this: a value is fully determined by the functions that can act on it (its representable presheaf in the category of types).

Principle 2: Mutation Is Local and Explicit

When mutation is needed (performance-critical loops, resource handles, FFI buffers), JAPL provides it through two mechanisms: (a) process-local state via recursion or the State[s] effect, and (b) linear/ownership-tracked mutable cells in the Resource layer. Mutation never leaks into the pure layer silently.

-- Process-local state via recursion (pure, no mutation)
fn counter(n: Int, acc: Int) -> Int =
  if n == 0 then acc
  else counter(n - 1, acc + n)

-- Explicit mutable state via effect
fn accumulate(items: List[Int]) -> Int with State[Int] =
  List.each items (fn x -> State.modify (fn acc -> acc + x))
  State.get()

-- Resource-layer mutable buffer (ownership tracked)
fn fill_buffer() -> Bytes with Io =
  let buf = Buffer.alloc(1024)   -- linear resource
  Buffer.write(buf, 0, data)
  Buffer.freeze(buf)              -- converts to immutable Bytes

Principle 3: Concurrency Is Process-Based, Not Shared-Memory-First

JAPL uses Erlang-style lightweight processes as the unit of concurrency. Processes do not share memory. They communicate through typed message passing. This model scales from a single core to a distributed cluster without changing the programming model.

type CounterMsg =
  | Increment
  | Decrement
  | GetCount(Reply[Int])

fn counter_process(count: Int) -> Never with Process[CounterMsg] =
  match Process.receive() with
  | Increment -> counter_process(count + 1)
  | Decrement -> counter_process(count - 1)
  | GetCount(reply) ->
      Reply.send(reply, count)
      counter_process(count)

-- Spawning a process
let pid = Process.spawn(fn -> counter_process(0))
Process.send(pid, Increment)

Principle 4: Failures Are Normal and Typed

JAPL treats failure as a first-class concern with two complementary strategies:

  1. Result types (from Rust): For expected, recoverable failures -- file not found, parse error, validation failure. These are values that must be handled.
  2. Crash/restart semantics (from Erlang): For unexpected failures -- hardware faults, corrupted state, bugs. Processes crash; supervisors restart them.
-- Result types for expected failures
type ParseError =
  | InvalidSyntax(String, Int)
  | UnexpectedEof

fn parse(input: String) -> Result[Ast, ParseError] =
  -- ...

-- Pattern matching forces handling
match parse(source) with
| Ok(ast) -> compile(ast)
| Err(InvalidSyntax(msg, line)) -> report_error(msg, line)
| Err(UnexpectedEof) -> report_error("unexpected end of file", 0)

-- Crash semantics for unexpected failures
fn database_worker(conn: DbConn) -> Never with Process[DbQuery] =
  -- If this crashes, the supervisor restarts it with a fresh connection
  let query = Process.receive()
  let result = Db.execute(conn, query)  -- may crash on connection loss
  Process.send(query.reply_to, result)
  database_worker(conn)

Principle 5: Distribution Is a Native Language Concern

Process IDs in JAPL are location-transparent. A PID may refer to a process on the local node or a remote node. The runtime handles serialization, discovery, and network transport. Types drive serialization: if a message type is serializable, it can cross node boundaries.

-- Connect to a remote node
let remote = Node.connect("worker-2.cluster.local:9000")

-- Spawn a process on a remote node
let pid = Process.spawn_on(remote, fn -> image_processor())

-- Send a message -- same syntax regardless of location
Process.send(pid, ProcessImage(image_data))

-- Service discovery
let registry = Registry.connect("service-registry.local")
let workers = Registry.lookup(registry, "image-processor")

Principle 6: The Default Unit of Composition Is the Function

Functions are the primary building block. Not classes, not objects, not components. Functions take values, return values, and compose. Modules group related functions. Traits define shared interfaces. There is no inheritance hierarchy.

-- Functions compose naturally
let process_order =
  validate_order
  >> calculate_totals
  >> apply_discounts
  >> generate_invoice

-- Pipelines for readability
let result =
  raw_data
  |> parse_csv
  |> List.filter(fn row -> row.amount > 0)
  |> List.map(to_transaction)
  |> List.sort_by(fn t -> t.date)

-- Modules group related functions
module Json =
  fn parse(input: String) -> Result[JsonValue, ParseError] = ...
  fn encode(value: JsonValue) -> String = ...
  fn pretty_print(value: JsonValue, indent: Int) -> String = ...

Principle 7: Runtime Simplicity Matters as Much as Type Power

A powerful type system is useless if it produces unpredictable runtime behavior. JAPL's runtime is designed to be simple, inspectable, and predictable:

  • No hidden allocations from autoboxing
  • No implicit conversions
  • Deterministic process scheduling semantics
  • Straightforward GC for immutable data, ownership for resources
  • Built-in observability: process introspection, message queue inspection, supervision tree visualization
-- Runtime introspection is built in
let info = Process.info(pid)
-- { status: Running, message_queue_len: 3, memory: 4096, ... }

let tree = Supervisor.which_children(sup)
-- Visual tree of all supervised processes and their states

Language Design Areas (20 Sections)

1. Syntax Style

JAPL follows the Go/Elm/Gleam tradition of minimal, readable syntax. There are no classes, no inheritance, no method overloading, no operator overloading beyond basic arithmetic.

-- Module declaration
module Http.Server

-- Import
import Http.{Request, Response, Status}
import Json

-- Type alias
type alias Headers = Map[String, String]

-- Function declaration
fn handle_request(req: Request) -> Response with Io, Net =
  let body = Json.parse(req.body)
  match body with
  | Ok(data) -> Response.json(Status.Ok, data)
  | Err(_) -> Response.text(Status.BadRequest, "invalid JSON")

-- No semicolons, no braces for blocks, indentation-aware
-- No classes, no `this`/`self`, no method dispatch

Design choice: Syntax should disappear. The reader should see logic, not ceremony.

2. Type System

JAPL's type system includes:

  • Algebraic data types (sum types and product types)
  • Parametric polymorphism (generics)
  • Local type inference (Hindley-Milner-style with extensions)
  • Traits (type classes, not OO interfaces)
  • Row-polymorphic records
  • Capability types for resource access
  • Lightweight effect typing (see Section 4)
-- Sum type (tagged union)
type Shape =
  | Circle(Float)
  | Rectangle(Float, Float)
  | Triangle(Float, Float, Float)

-- Parametric polymorphism
fn map(list: List[a], f: fn(a) -> b) -> List[b] =
  match list with
  | [] -> []
  | [x, ..rest] -> [f(x), ..map(rest, f)]

-- Row-polymorphic record
fn get_name(r: { name: String | rest }) -> String =
  r.name

-- This works for ANY record that has a `name: String` field
let _ = get_name({ name = "Alice", age = 30 })
let _ = get_name({ name = "Bob", email = "[email protected]" })

-- Trait definition
trait Show[a] =
  fn show(value: a) -> String

-- Trait implementation
impl Show[Shape] =
  fn show(shape) =
    match shape with
    | Circle(r) -> "Circle(" ++ Float.to_string(r) ++ ")"
    | Rectangle(w, h) -> "Rectangle(" ++ Float.to_string(w) ++ ", " ++ Float.to_string(h) ++ ")"
    | Triangle(a, b, c) -> "Triangle(...)"

-- Capability type
type FileHandle = capability {
  read: fn(Int) -> Bytes with Io,
  write: fn(Bytes) -> Result[Int, IoError] with Io,
  close: fn() -> Unit with Io,
}

3. Ownership Model

JAPL has two layers with distinct memory disciplines:

Pure Layer: Immutable data managed by a garbage collector. Values are freely shared and copied (structurally). This is the default layer where most code lives.

Resource Layer: Mutable resources (file handles, network sockets, GPU buffers, FFI pointers) managed by linear types and ownership tracking. Resources must be consumed exactly once: used and then released.

-- Pure layer: GC-managed, immutable, freely shareable
let data = [1, 2, 3, 4, 5]
let copy = data  -- sharing is fine, data is immutable

-- Resource layer: ownership-tracked, must be consumed
fn process_file(path: String) -> Result[String, IoError] with Io =
  -- `use` binds a linear resource that must be consumed
  use file = File.open(path, Read)?
  let contents = File.read_all(file)?
  File.close(file)  -- consumed here; forgetting this is a compile error
  Ok(contents)

-- Transfer ownership
fn send_to_worker(buf: own Buffer, pid: Pid[WorkerMsg]) -> Unit =
  Process.send(pid, ProcessBuffer(buf))
  -- `buf` is moved; using it here is a compile error

-- Borrow (read-only reference, does not consume)
fn peek(buf: ref Buffer) -> Byte =
  Buffer.get(buf, 0)

4. Effects

JAPL tracks computational effects in function signatures. Effects compose and propagate. The base effects are:

Effect Meaning
Pure No effects (default, not written)
Io File system, console, clock, random
Async Asynchronous operations
Net Network access
State[s] Mutable state of type s
Process Process spawn/send/receive
Fail[e] May fail with error type e
-- Pure function: no annotation needed
fn add(a: Int, b: Int) -> Int =
  a + b

-- Effectful function: effects listed after `with`
fn read_config(path: String) -> Config with Io, Fail[ConfigError] =
  let text = File.read_to_string(path)?
  parse_config(text)?

-- Effects compose: calling effectful functions in a pure context is a type error
fn handler(req: Request) -> Response with Io, Net, Fail[AppError] =
  let config = read_config("/etc/app.conf")?    -- adds Io, Fail[ConfigError]
  let data = Http.get(config.data_url)?          -- adds Net
  process_response(data)

-- Effect handlers: run effectful code, providing the effect implementation
fn main() -> Unit with Io =
  let result = State.run(0, fn ->
    accumulate([1, 2, 3, 4, 5])
  )
  Io.println(Int.to_string(result))

Effects are compositional annotations. A function's effect signature is the union of all effects it (transitively) invokes. The compiler infers effects locally and checks them at module boundaries.

5. Concurrency

JAPL processes are lightweight (thousands to millions per node), isolated (no shared memory), and scheduled cooperatively by the runtime. Each process has a typed mailbox.

-- Define message types
type WorkerMsg =
  | DoWork(Task, Reply[TaskResult])
  | Shutdown

-- Define a worker process
fn worker(state: WorkerState) -> Never with Process[WorkerMsg] =
  match Process.receive() with
  | DoWork(task, reply) ->
      let result = execute_task(state, task)
      Reply.send(reply, result)
      worker(state)
  | Shutdown ->
      cleanup(state)
      Process.exit(Normal)

-- Typed mailbox prevents sending wrong message types
let pid: Pid[WorkerMsg] = Process.spawn(fn -> worker(initial_state))
Process.send(pid, DoWork(my_task, reply_channel))

-- Selective receive with timeout
fn wait_for_response(timeout_ms: Int) -> Result[Response, Timeout] with Process[Msg] =
  Process.receive_with_timeout(timeout_ms) {
  | DataReady(data) -> Ok(process(data))
  | _ -> Err(Timeout)
  }

6. Supervision Trees

Supervision is built into the language and runtime. Supervisors are processes that monitor child processes and restart them according to a strategy when they fail.

type RestartStrategy =
  | OneForOne      -- restart only the failed child
  | AllForOne      -- restart all children if one fails
  | RestForOne     -- restart the failed child and all children started after it

type ChildSpec =
  { id: String
  , start: fn() -> Never
  , restart: Permanent | Transient | Temporary
  , shutdown: Timeout(Int) | Brutal
  }

fn start_app() -> Pid[SupervisorMsg] with Process =
  Supervisor.start(
    strategy = OneForOne,
    max_restarts = 5,
    max_seconds = 60,
    children = [
      { id = "db_pool"
      , start = fn -> DbPool.start(config.database)
      , restart = Permanent
      , shutdown = Timeout(5000)
      },
      { id = "http_server"
      , start = fn -> HttpServer.start(config.http)
      , restart = Permanent
      , shutdown = Timeout(10000)
      },
      { id = "background_jobs"
      , start = fn -> JobRunner.start(config.jobs)
      , restart = Transient
      , shutdown = Timeout(30000)
      },
    ]
  )

7. Error Handling

Two complementary error strategies:

Result types for expected, domain-level failures:

type AppError =
  | NotFound(String)
  | Unauthorized
  | ValidationError(List[FieldError])
  | DatabaseError(DbError)

fn get_user(id: UserId) -> Result[User, AppError] with Io =
  let row = Db.query_one(sql, [id])? |> map_err(DatabaseError)?
  validate_user(row)?
  Ok(to_user(row))

-- The `?` operator propagates errors, analogous to Rust
-- Equivalent to:
--   match Db.query_one(sql, [id]) with
--   | Ok(row) -> ...
--   | Err(e) -> Err(DatabaseError(e))

Crash/restart for unexpected failures:

-- A process that crashes is restarted by its supervisor
-- No need to handle every possible failure in the process itself
fn tcp_acceptor(listener: TcpListener) -> Never with Process[AcceptorMsg], Io =
  let conn = Tcp.accept(listener)  -- may crash on OS error
  let pid = Process.spawn(fn -> handle_connection(conn))
  Process.link(pid)                -- link so we know if handler crashes
  tcp_acceptor(listener)           -- loop; if WE crash, supervisor restarts us

8. Distribution

Distribution is native. Process IDs are location-transparent. The runtime handles marshaling, connection management, and node discovery.

-- Node configuration
let node = Node.start(
  name = "web-1",
  cookie = Env.get("CLUSTER_COOKIE"),
  listen = "0.0.0.0:9000",
)

-- Remote process spawn
let remote_node = Node.connect("worker-1.internal:9000")
let pid = Process.spawn_on(remote_node, fn ->
  heavy_computation(params)
)

-- Monitor remote process
Process.monitor(pid)

-- Receive results (same syntax as local)
match Process.receive() with
| ComputationResult(data) -> handle_result(data)
| ProcessDown(^pid, reason) -> handle_failure(reason)

-- Type-derived serialization: any type that derives Serialize
-- can cross node boundaries
type JobRequest deriving(Serialize, Deserialize) =
  { id: JobId
  , payload: Bytes
  , priority: Priority
  }

9. Go-Like Practicality

JAPL ships with a unified toolchain:

  • japl build -- fast incremental compilation to static binaries
  • japl run -- compile and execute
  • japl test -- built-in test runner with property-based testing support
  • japl fmt -- opinionated formatter (one true style)
  • japl doc -- documentation generator from doc comments
  • japl deps -- dependency management
  • japl release -- cross-compilation and release packaging
-- Built-in test syntax
test "parsing a valid integer" =
  assert parse_int("42") == Ok(42)

test "parsing rejects non-numeric input" =
  assert parse_int("abc") == Err(InvalidInt("abc"))

-- Property-based testing
property "reversing a list twice is identity" =
  forall (xs: List[Int]) ->
    List.reverse(List.reverse(xs)) == xs

-- Benchmarks
bench "fibonacci 30" =
  fibonacci(30)

Compilation targets: native (via LLVM or Cranelift), WASM. Cross-compilation produces static binaries with no external dependencies.

10. Memory Model

Immutable Heap (GC-managed):

  • All pure values live here: ADTs, records, closures, strings, lists
  • Generational, concurrent garbage collector
  • No write barriers needed for immutable data (simplified GC)
  • Safe to share across processes (no mutation, no races)

Resource Arena (Ownership-managed):

  • Mutable resources: buffers, handles, connections, FFI pointers
  • Linear types ensure exactly-once consumption
  • No GC involvement; deterministic cleanup at scope exit
  • Explicit transfer of ownership between processes
-- The compiler statically determines which layer each value lives in

-- Immutable heap (automatic)
let config = { host = "localhost", port = 8080 }

-- Resource arena (explicit)
use socket = Tcp.connect(config.host, config.port)?
let response = Tcp.send(socket, request)?
Tcp.close(socket)  -- deterministic cleanup

11. FFI

JAPL provides FFI to C, Rust, and WASM. All FFI is capability-wrapped: you must hold an Unsafe capability to call foreign functions, and the effect system tracks this.

-- Declare a foreign function
foreign "C" fn sqlite3_open(filename: CString, db: Ptr[Ptr[Sqlite3]]) -> CInt

-- Wrap in a safe JAPL interface
fn open_database(path: String) -> Result[Database, DbError] with Io =
  use filename = CString.from(path)
  use db_ptr = Ptr.alloc[Ptr[Sqlite3]]()
  let rc = unsafe sqlite3_open(filename, db_ptr)
  if rc == 0 then
    Ok(Database.from_raw(Ptr.read(db_ptr)))
  else
    Err(DbError.from_code(rc))

-- WASM interop
foreign "wasm" module "image_codec" =
  fn encode_png(data: Bytes, width: Int, height: Int) -> Bytes
  fn decode_png(data: Bytes) -> Result[Image, CodecError]

12. Module System

Modules are namespaces for types and functions. Signatures (module types) define interfaces. Traits define shared behavior across types. There is no OO identity commitment -- no self, no method dispatch tables.

-- Module definition
module Map

-- Opaque type (implementation hidden)
opaque type Map[k, v]

-- Public interface
fn empty() -> Map[k, v]
fn insert(map: Map[k, v], key: k, value: v) -> Map[k, v] where Ord[k]
fn lookup(map: Map[k, v], key: k) -> Option[v] where Ord[k]
fn delete(map: Map[k, v], key: k) -> Map[k, v] where Ord[k]
fn fold(map: Map[k, v], init: acc, f: fn(acc, k, v) -> acc) -> acc

-- Module signature (interface)
signature KeyValueStore[k, v] =
  type Store
  fn create() -> Store with Io
  fn get(store: Store, key: k) -> Option[v] with Io
  fn set(store: Store, key: k, value: v) -> Unit with Io
  fn delete(store: Store, key: k) -> Unit with Io

-- Module implementing a signature
module RedisStore : KeyValueStore[String, String] =
  type Store = RedisConnection
  fn create() -> Store with Io = Redis.connect(default_config)
  fn get(store, key) with Io = Redis.get(store, key)
  fn set(store, key, value) with Io = Redis.set(store, key, value)
  fn delete(store, key) with Io = Redis.del(store, key)

13. Explicit State

State is never implicit. Three mechanisms for managing state:

-- 1. Process-local state via recursion (most common)
fn server_loop(state: ServerState) -> Never with Process[ServerMsg] =
  let msg = Process.receive()
  let new_state = handle_message(state, msg)
  server_loop(new_state)

-- 2. State effect (for single-threaded stateful computation)
fn parser(input: String) -> Ast with State[ParserState], Fail[ParseError] =
  State.put({ input = input, pos = 0, errors = [] })
  let ast = parse_expression()
  let final_state = State.get()
  if List.is_empty(final_state.errors) then ast
  else Fail.raise(ParseError(final_state.errors))

-- 3. Linear mutable cells (for performance-critical code)
fn sort_in_place(arr: own MutableArray[Int]) -> own MutableArray[Int] =
  let len = MutableArray.length(arr)
  -- imperative loop with owned mutable array
  loop i = 0, arr = arr while i < len do
    loop j = i + 1, arr = arr while j < len do
      if MutableArray.get(arr, j) < MutableArray.get(arr, i) then
        let arr = MutableArray.swap(arr, i, j)
        continue(j + 1, arr)
      else
        continue(j + 1, arr)
    continue(i + 1, arr)
  arr

14. Data-Oriented Design

JAPL supports data-oriented patterns for cache-friendly, high-performance code:

-- Packed struct layout (contiguous memory, no pointers)
type Vec3 = packed { x: Float32, y: Float32, z: Float32 }

-- Tagged unions have efficient representations
type Particle = packed
  | Active(Vec3, Vec3, Float32)   -- position, velocity, mass
  | Inactive

-- Structural records (anonymous, row-polymorphic)
let point = { x = 1.0, y = 2.0 }

-- Zero-copy byte slicing
fn parse_header(data: Bytes) -> Header with Fail[ParseError] =
  let magic = Bytes.slice(data, 0, 4)    -- no copy, just a view
  let version = Bytes.read_u32_be(data, 4)
  let length = Bytes.read_u32_be(data, 8)
  { magic, version, length }

15. Web/Backend Ergonomics

HTTP handlers, JSON processing, and database access are first-class concerns with clean effect annotations:

fn routes() -> Router =
  Router.new()
  |> Router.get("/users/:id", get_user_handler)
  |> Router.post("/users", create_user_handler)
  |> Router.delete("/users/:id", delete_user_handler)

fn get_user_handler(req: Request) -> Response with Io, Net, Fail[AppError] =
  let id = Request.param(req, "id") |> parse_user_id?
  let user = UserRepo.find_by_id(id)?
  match user with
  | Some(u) -> Response.json(200, User.to_json(u))
  | None -> Response.json(404, Json.object([("error", Json.string("not found"))]))

fn create_user_handler(req: Request) -> Response with Io, Net, Fail[AppError] =
  let body = Request.json_body(req)?
  let params = CreateUserParams.from_json(body)?
  let user = UserRepo.create(params)?
  Response.json(201, User.to_json(user))

16. Actor + Function Fusion

JAPL unifies two paradigms:

  • Functions transform values (the logic layer)
  • Processes own time and state (the concurrency layer)
  • Messages connect processes (the communication layer)
-- Pure function: transforms data
fn calculate_price(item: Item, quantity: Int, discount: Discount) -> Money =
  let base = Money.multiply(item.price, quantity)
  Discount.apply(discount, base)

-- Process: owns state over time, delegates logic to functions
fn order_processor(state: OrderState) -> Never with Process[OrderMsg] =
  match Process.receive() with
  | NewOrder(order) ->
      let priced = List.map(order.items, fn item ->
        { item | total = calculate_price(item, item.qty, order.discount) }
      )
      let new_state = { state | pending = Map.insert(state.pending, order.id, priced) }
      order_processor(new_state)
  | ConfirmOrder(id, reply) ->
      match Map.lookup(state.pending, id) with
      | Some(order) ->
          Reply.send(reply, Ok(order))
          order_processor({ state | pending = Map.delete(state.pending, id) })
      | None ->
          Reply.send(reply, Err(OrderNotFound(id)))
          order_processor(state)

17. What to Avoid

JAPL deliberately excludes:

Excluded Feature Reason
Inheritance Composition via functions and traits is simpler and more flexible
Null / nil Option[a] is explicit and checked at compile time
Ambient exceptions Effects and Result types make failure explicit and trackable
Shared mutable memory Process isolation eliminates data races by construction
Monad jargon Effects are described by what they DO, not by their categorical name
Implicit conversions Explicit is always clearer than implicit
Method overloading One name, one meaning
Macros (initially) Keep the language simple; revisit if proven necessary

18. Slogan

"Pure by default, concurrent by design, resource-safe by construction, distributed without apology."

Each clause maps to a core design decision:

  • Pure by default -- immutable values, no side effects without annotation
  • Concurrent by design -- processes and message passing are language primitives
  • Resource-safe by construction -- ownership types prevent resource leaks at compile time
  • Distributed without apology -- location-transparent PIDs, native clustering, type-derived serialization

19. Minimal Feature Summary

Category Feature
Values Immutable ADTs, records, tuples, lists, maps
Functions First-class, closures, pattern matching, pipelines
Types ADTs, generics, traits, row polymorphism, local inference
Effects Io, Async, Net, State, Process, Fail -- tracked in types
Resources Linear types, ownership, borrowing, deterministic cleanup
Concurrency Lightweight processes, typed mailboxes, monitors, links
Fault tolerance Supervisors, restart strategies, crash recovery
Distribution Location-transparent PIDs, node clustering, service discovery
Tooling Compiler, formatter, test runner, build tool, cross-compilation

20. One Sentence

"Pure functions handle logic, supervised processes handle time and failure, and ownership handles resources."


Type System Specification

Primitive Types

Int          -- arbitrary precision integer
Float        -- 64-bit IEEE 754
Float32      -- 32-bit IEEE 754
Bool         -- True | False
Char         -- Unicode scalar value
String       -- UTF-8 encoded, immutable
Bytes        -- raw byte sequence, immutable
Unit         -- the unit type, single value ()
Never        -- the bottom type, no values (for non-terminating processes)

Algebraic Data Types

-- Sum types (tagged unions)
type Option[a] =
  | Some(a)
  | None

type Result[a, e] =
  | Ok(a)
  | Err(e)

type List[a] =
  | Cons(a, List[a])
  | Nil

-- Product types (records)
type User =
  { id: UserId
  , name: String
  , email: String
  , created_at: Timestamp
  }

Type Inference

JAPL uses bidirectional type checking with local inference. Top-level functions require signatures at module boundaries; within function bodies, types are inferred.

-- Signature required at module boundary
fn process(items: List[Item]) -> Summary with Io =
  -- Types inferred within the body
  let totals = List.map(items, fn item -> item.price * item.quantity)
  let sum = List.fold(totals, 0, fn acc, t -> acc + t)
  { item_count = List.length(items), total = sum }

Traits (Type Classes)

trait Eq[a] =
  fn eq(x: a, y: a) -> Bool

trait Ord[a] where Eq[a] =
  fn compare(x: a, y: a) -> Ordering

trait Functor[f] =
  fn map(fa: f[a], func: fn(a) -> b) -> f[b]

trait Serialize[a] =
  fn serialize(value: a) -> Bytes
  fn deserialize(data: Bytes) -> Result[a, SerializeError]

-- Deriving
type Point deriving(Eq, Ord, Show, Serialize) =
  { x: Float, y: Float }

Row Polymorphism

Records are structurally typed with row polymorphism, enabling duck-typing safety:

-- This function works on ANY record with a `name` field
fn greet(person: { name: String | r }) -> String =
  "Hello, " ++ person.name ++ "!"

-- All of these work:
greet({ name = "Alice" })
greet({ name = "Bob", age = 30 })
greet({ name = "Charlie", role = Admin, email = "[email protected]" })

Capability Types

Capabilities are types that represent permission to perform an action. They cannot be forged; they must be granted by the runtime or a parent process.

type FsCapability = capability { root: Path, permissions: FsPermissions }

fn read_file(cap: FsCapability, path: Path) -> Result[String, IoError] with Io =
  if Path.is_within(path, cap.root) && cap.permissions.read then
    File.read_to_string(path)
  else
    Err(PermissionDenied)

Effect System Specification

Effect Algebra

Effects form a commutative monoid under composition:

  • Identity: Pure (no effects)
  • Composition: with E1, E2 is the union of effects
  • Commutativity: with Io, Net = with Net, Io
  • Idempotency: with Io, Io = with Io

Effect Hierarchy

Pure < State[s] < Io
Pure < Fail[e]
Pure < Process < Async
Pure < Net < Io

A function with effect E1 can call functions with effect E2 only if E2 <= E1 in the hierarchy, or if E2 is listed in the caller's effect set.

Effect Handlers

-- Running a State effect
let result: Int = State.run(initial_state, fn ->
  stateful_computation()
)

-- Running a Fail effect
let result: Result[a, e] = Fail.catch(fn ->
  fallible_computation()
)

-- Effects can be handled at any level
fn main() -> Unit with Io =
  let result = Fail.catch(fn ->
    State.run(0, fn ->
      complex_computation()
    )
  )
  match result with
  | Ok(value) -> Io.println("Success: " ++ show(value))
  | Err(e) -> Io.println("Error: " ++ show(e))

Effect Inference

Within a function body, effects are inferred from the calls made. At module boundaries, effect signatures are checked.

-- The compiler infers this function has effects: Io, Fail[ParseError]
fn load_config(path: String) -> Config with Io, Fail[ParseError] =
  let text = File.read_to_string(path)   -- infers Io
  parse_config(text)                       -- infers Fail[ParseError]

Categorical Interpretation

Effects in JAPL correspond to functors in the categorical sense. The effect system forms a graded monad where:

  • Each effect E defines a functor T_E: Type -> Type
  • Effect composition corresponds to functor composition
  • Effect handlers correspond to natural transformations T_E => Id (interpreting the effect away)

This connects to the YonedaAI framework: the Yoneda lemma guarantees that any natural transformation between effect functors is uniquely determined by its action on a single element, which is the theoretical basis for effect handler correctness.


Process / Concurrency Model

Process Lifecycle

Spawned -> Running -> (Waiting | Running) -> Exited(reason)

Processes are:

  • Lightweight: stack starts small, grows as needed. Target: millions of processes per node.
  • Isolated: each process has its own heap partition. No shared mutable state.
  • Preemptively scheduled: the runtime scheduler ensures fairness via reduction counting.
  • Garbage-collected independently: GC for one process does not pause others.

Communication Primitives

-- Spawn
let pid: Pid[Msg] = Process.spawn(fn -> process_body())

-- Send (async, non-blocking, always succeeds for local processes)
Process.send(pid, message)

-- Receive (blocking, with optional timeout)
let msg = Process.receive()
let msg = Process.receive_with_timeout(5000)

-- Selective receive
let msg = Process.receive_matching(fn msg ->
  match msg with
  | Priority(High, _) -> True
  | _ -> False
)

-- Links (bidirectional failure propagation)
Process.link(pid)

-- Monitors (unidirectional failure observation)
let ref = Process.monitor(pid)
-- Receives ProcessDown(ref, pid, reason) when monitored process exits

Typed Mailboxes

Each process has a single mailbox typed by its message type. This prevents sending incompatible messages:

type LoggerMsg =
  | Log(Level, String)
  | Flush(Reply[Unit])
  | SetLevel(Level)

-- This process can only receive LoggerMsg
fn logger(config: LogConfig) -> Never with Process[LoggerMsg] =
  match Process.receive() with
  | Log(level, msg) ->
      if level >= config.min_level then
        write_log(config.output, level, msg)
      logger(config)
  | Flush(reply) ->
      flush_output(config.output)
      Reply.send(reply, ())
      logger(config)
  | SetLevel(level) ->
      logger({ config | min_level = level })

Supervision

            Application Supervisor
           /          |           \
     DB Pool      HTTP Server    Job Runner
     /    \        /    \            |
  Conn1  Conn2  Acc1   Acc2     Worker Pool
                                /    |    \
                             W1     W2     W3

Supervision strategies:

  • OneForOne: Only the crashed child is restarted.
  • AllForOne: All children are restarted when one crashes.
  • RestForOne: The crashed child and all children started after it are restarted.

Restart intensity: max_restarts within max_seconds. If exceeded, the supervisor itself crashes (propagating up the tree).

Process Algebra Connection

JAPL's process model can be formalized using process algebra (CCS/CSP/pi-calculus). This connects to the YonedaAI research on categorical models of concurrency:

  • Processes correspond to objects in a process category
  • Messages correspond to morphisms
  • Parallel composition is a monoidal product
  • Communication channels are profunctors between process categories

Memory Model

Two-Layer Architecture

+--------------------------------------------------+
|                  JAPL Process                      |
|                                                    |
|  +-------------------+  +----------------------+  |
|  |   Immutable Heap   |  |   Resource Arena     |  |
|  |   (GC-managed)     |  |   (Ownership-managed)|  |
|  |                     |  |                      |  |
|  |  - ADT values       |  |  - File handles      |  |
|  |  - Records          |  |  - Sockets           |  |
|  |  - Closures         |  |  - Buffers           |  |
|  |  - Strings          |  |  - FFI pointers      |  |
|  |  - Lists/Maps       |  |  - GPU resources     |  |
|  |                     |  |                      |  |
|  |  Freely shareable   |  |  Single owner        |  |
|  |  across processes   |  |  Deterministic free  |  |
|  +-------------------+  +----------------------+  |
+--------------------------------------------------+

GC Strategy for Immutable Heap

  • Generational collector: most values are short-lived
  • Per-process heaps: GC in one process does not stop others
  • No write barriers: immutable data has no references that change
  • Concurrent major collection for long-lived data
  • Process death reclaims entire heap instantly

Ownership Rules for Resource Arena

  1. Every resource has exactly one owner at any time
  2. Ownership can be transferred (own qualifier) but not duplicated
  3. Read-only borrows (ref qualifier) allow temporary shared access
  4. When the owner goes out of scope, the resource is deterministically freed
  5. The compiler rejects programs that use a resource after transfer or free
-- Resource lifecycle
fn example() -> Unit with Io =
  use conn = Db.connect(url)?        -- conn is owned here
  let result = Db.query(ref conn, sql)?  -- borrow for query
  Db.close(conn)                     -- ownership consumed, resource freed
  -- conn cannot be used here: compile error

Research Program -- 7 Papers

Each of JAPL's 7 core principles receives a dedicated research paper examining its theoretical foundations, practical trade-offs, and formal semantics.

Paper 1: Values Are Primary -- Immutability as a Foundation for Concurrent Systems

Theoretical foundations:

  • Persistent data structures and structural sharing
  • The connection between immutability and referential transparency
  • Category-theoretic view: values as objects in a Cartesian closed category

Comparison: Haskell (lazy, pure), Clojure (persistent collections), Erlang (immutable terms), Rust (move semantics), Go (value types but mutable)

JAPL contribution: Demonstrates that default immutability combined with per-process GC eliminates the need for concurrent GC while retaining ergonomic memory management.

Paper 2: Mutation Is Local and Explicit -- Controlled Effects in a Functional Setting

Theoretical foundations:

  • Effect systems and coeffect systems
  • Graded monads and algebraic effects
  • Linear logic and its connection to resource management

Comparison: Haskell (IO monad), OCaml (unrestricted mutation), Rust (mut keyword + borrow checker), Koka (effect handlers), Frank (effect types)

JAPL contribution: Shows how a layered effect system (pure/state/io) combined with linear types for the resource layer provides both safety and performance without monadic syntax overhead.

Paper 3: Concurrency Is Process-Based -- Isolation as a Concurrency Primitive

Theoretical foundations:

  • Process calculi (pi-calculus, CSP, CCS)
  • Actor model formalization
  • Session types and behavioral types

Comparison: Erlang/OTP (gold standard for process isolation), Go (goroutines + channels, shared memory possible), Rust (threads + ownership), Haskell (STM + forkIO), Pony (reference capabilities)

JAPL contribution: Formalizes the connection between typed mailboxes and session types, showing that JAPL's process model provides deadlock-freedom guarantees for common patterns.

Paper 4: Failures Are Normal and Typed -- A Unified Theory of Error Handling

Theoretical foundations:

  • Exception semantics in typed lambda calculus
  • Supervision theory from control systems
  • Failure detectors in distributed systems (Chandra-Toueg)

Comparison: Rust (Result + panic), Erlang (let-it-crash), Go (error values), Haskell (Either + exceptions), Java (checked + unchecked exceptions)

JAPL contribution: Proves that the combination of Result types and supervision trees covers all failure modes with no gaps: expected failures are handled as values, unexpected failures are handled by restarting.

Paper 5: Distribution Is a Native Concern -- Location Transparency in a Typed Setting

Theoretical foundations:

  • Distribution transparency (Waldo et al., "A Note on Distributed Computing")
  • Type-safe serialization and marshaling
  • Consensus and coordination in distributed systems

Comparison: Erlang (distribution built in), Akka (location-transparent actors), Go (gRPC + manual), Orleans (virtual actors)

JAPL contribution: Addresses Waldo's critique by making distribution boundaries visible through effects (Net) while keeping PID-based communication uniform. Type-derived serialization eliminates protocol mismatches.

Paper 6: Functions as the Unit of Composition -- Categorical Semantics of Modular Design

Theoretical foundations:

  • Category theory: categories, functors, natural transformations
  • Yoneda lemma applied to type theory
  • Module systems (ML modules, Backpack)
  • Profunctors and optics for composable data access

Comparison: Haskell (type classes + modules), OCaml (ML modules + functors), Rust (traits + crates), Go (packages + interfaces), Elm (modules)

JAPL contribution: Uses the Yoneda perspective to design a module system where a module is characterized by its signature (the functions it exposes), providing a rigorous foundation for separate compilation and modular reasoning.

Paper 7: Runtime Simplicity -- Predictable Execution for Functional Languages

Theoretical foundations:

  • Abstract machine semantics (CESK, STG)
  • Cost semantics for functional languages
  • Real-time garbage collection
  • Work-stealing schedulers

Comparison: Go (simple runtime, goroutine scheduler), Erlang (BEAM VM, reduction counting), Haskell (STG machine, lazy evaluation complexity), OCaml (simple native compilation), Rust (no runtime)

JAPL contribution: Defines a cost model where every JAPL expression has a bounded and predictable cost, enabling performance reasoning without understanding compiler optimizations.


Cross-Reference Map: YonedaAI Math to JAPL PL Concepts

This section maps the mathematical frameworks developed in the YonedaAI research program to concrete programming language design decisions in JAPL.

Category Theory -> Module System and Composition

YonedaAI Concept JAPL PL Concept
Category Module (objects = types, morphisms = functions)
Functor Parametric type constructor (e.g., List, Option) + Functor trait
Natural transformation Polymorphic function between type constructors (e.g., fn to_list(opt: Option[a]) -> List[a])
Yoneda lemma A type is determined by the functions that can act on it -- structural typing, trait-based polymorphism
Representable presheaf A type's interface: the set of all functions from that type
Cartesian closed category JAPL's type system: product types (records), function types (closures)

Explanation: The Yoneda lemma states that an object in a category is uniquely determined (up to isomorphism) by its relationships to all other objects. In JAPL, this manifests as: a value's type is determined by the functions (trait implementations) available for it. Row-polymorphic records embody this: { name: String | r } says "I am any type that relates to String via a name accessor."

Emergence Functors -> Effect Handlers

YonedaAI Concept JAPL PL Concept
Emergence functor (non-faithful, non-full) Effect handler: maps effectful computation to pure result, losing information about HOW effects were performed
Emergence kernel Effect abstraction: distinct effectful programs that produce the same pure result
Measurement opacity Abstraction barrier: code using an effect handler cannot observe the handler's internal implementation
Indirect witnesses Effect-polymorphic functions: can observe SOME properties of effects without full access

Self-Reference Incompleteness -> Type System Boundaries

YonedaAI Concept JAPL PL Concept
SRIP (Self-Reference Incompleteness Principle) A type system cannot fully type-check itself (Godel-like limitation)
Lawvere fixed-point theorem No type T such that T = T -> Bool (Russell-like paradox prevented by positivity checking)
Diagonal construction Recursive types require explicit rec annotation to prevent non-termination
Emergence Incompleteness FFI boundary: the type system cannot verify properties of foreign code

Embedded Observer Constraint -> Process Isolation

YonedaAI Concept JAPL PL Concept
Embedded Observer Constraint Each process can only observe the system through its mailbox (its "measurement apparatus")
Epistemic horizon A process's typed mailbox defines what it can observe: messages of its declared type
Accessible region R|_S The set of messages a process can receive and the replies it can obtain
Measurement presheaf failure (non-sheaf) Distributed processes may have inconsistent views (no global consistent snapshot)
Extension deficit (Kan extensions) The gap between what a process can infer about the system and the actual system state

Kan Extensions -> Type Inference and Default Implementations

YonedaAI Concept JAPL PL Concept
Left Kan extension Best possible type inference: given partial type information, infer the most general type
Right Kan extension Most restrictive constraint propagation: given requirements, compute the tightest bound
Extension deficit Cases where type inference is incomplete and explicit annotation is required

Decoherence Functor -> Compilation / Erasure

YonedaAI Concept JAPL PL Concept
Decoherence functor Compilation: maps richly-typed source language to simpler runtime representation
Presheaf coarsening Type erasure: runtime values carry less type information than compile-time types
Pointer states (fixed points of decoherence) Runtime representations that survive compilation unchanged (e.g., Int stays Int)
Classical limit as colimit Fully compiled code: the colimit of progressive compilation passes

Process Algebra -> Categorical Models of Concurrency

YonedaAI Concept JAPL PL Concept
Morphisms in measurement category Messages between processes
Monoidal product on categories Parallel process composition
Profunctors Bidirectional communication channels between process groups
Optics (lenses, prisms) Composable message routing and pattern matching on process messages

Key Design Decisions and Trade-offs

1. GC for Immutable Data vs. Full Ownership

Decision: Use GC for the immutable pure layer, ownership for the resource layer.

Trade-off: Simpler programming model (no lifetime annotations for most code) at the cost of GC pauses. Mitigated by per-process heaps (GC is local) and the fact that immutable data simplifies collection (no write barriers).

Alternatives considered: Full ownership everywhere (Rust-style) -- rejected because lifetime annotations create friction for high-level functional code. Full GC (Go-style) -- rejected because it cannot guarantee deterministic resource cleanup.

2. Effect Annotations vs. Unrestricted Side Effects

Decision: Effects are tracked in the type system.

Trade-off: More verbose function signatures in exchange for knowing, at any call site, what a function might do. Effect inference within function bodies reduces annotation burden.

Alternatives considered: Unrestricted effects (OCaml, Go) -- rejected because it makes reasoning about concurrency and distribution harder. Full monadic effects (Haskell) -- rejected because monad transformers are ergonomically painful.

3. Process Isolation vs. Shared Memory

Decision: No shared mutable memory between processes.

Trade-off: Message copying overhead for large data in exchange for elimination of data races. Mitigated by immutable data sharing (zero-copy for GC-managed values) and ownership transfer for resources.

Alternatives considered: Shared memory with locks (Java, Go) -- rejected as the primary model because it reintroduces data races. Software transactional memory (Haskell STM) -- potentially a future extension but not in the core model.

4. Typed Mailboxes vs. Untyped Messages

Decision: Each process has a single mailbox with a declared message type.

Trade-off: Less flexibility (cannot send arbitrary messages to any process) in exchange for compile-time safety. Processes that need to handle multiple message types use a sum type.

Alternatives considered: Untyped mailboxes (Erlang) -- rejected because type errors at runtime are harder to debug at scale. Multiple typed channels (Go-style) -- rejected because a single mailbox with a sum type is simpler and avoids select/channel management.

5. Strict Evaluation vs. Lazy Evaluation

Decision: Strict (eager) evaluation by default.

Trade-off: Predictable memory usage and performance at the cost of not having lazy infinite data structures built in. Lazy evaluation available explicitly via thunks.

Alternatives considered: Lazy by default (Haskell) -- rejected because space leaks and unpredictable performance profiles conflict with Principle 7 (runtime simplicity).

6. No Inheritance

Decision: No class inheritance, no method overriding, no subtype polymorphism via inheritance.

Trade-off: Some design patterns (e.g., template method, strategy via inheritance) require reformulation using functions and traits. The payoff is a simpler language with no fragile base class problem, no diamond inheritance, and no confusion between subtyping and inheritance.

7. Row Polymorphism vs. Nominal Records

Decision: Records are structurally typed with row polymorphism.

Trade-off: More flexible (functions work on any record with the right fields) but potentially weaker error messages when fields are misspelled. Nominal types (ADTs) are available for when identity matters.


Comparison Matrix

JAPL vs. Rust vs. Go vs. Erlang vs. Haskell vs. OCaml

Feature JAPL Rust Go Erlang Haskell OCaml
Paradigm Functional-first, process-concurrent Systems, multi-paradigm Imperative, concurrent Functional, concurrent Pure functional, lazy Functional, imperative
Immutability Default Opt-in (let vs let mut) Opt-in (const) Default Default Opt-in (mutable keyword)
Type system ADTs, traits, row poly, effects ADTs, traits, lifetimes Interfaces, structs Dynamic, pattern matching ADTs, type classes, HKTs ADTs, modules, functors
Type inference Local (HM + extensions) Local None (minimal) N/A (dynamic) Global (HM + extensions) Global (HM)
Effect tracking Yes (in types) Partial (unsafe, Send/Sync) No No Yes (IO monad) No
Null No (Option type) No (Option type) Yes (nil) No (pattern matching) No (Maybe type) No (option type)
Error handling Result + supervision Result + panic Error values + panic Pattern match + crash Either + exceptions Result + exceptions
Concurrency model Processes (Erlang-style) Threads + async/await Goroutines + channels Processes Threads + STM + async Threads + async (5.x)
Shared mutable state No (process isolation) Controlled (ownership) Yes (mutexes, channels) No (process isolation) No (STM for controlled) Yes (refs)
Memory management GC (pure) + ownership (resources) Ownership + borrowing GC GC GC GC
Supervision Built-in No No Built-in (OTP) No No
Distribution Built-in No No (manual gRPC etc.) Built-in No No
Pattern matching Yes (exhaustive) Yes (exhaustive) No (type switch) Yes (exhaustive) Yes (exhaustive) Yes (exhaustive)
Compilation Native + WASM Native + WASM Native BEAM bytecode Native (GHC) Native + bytecode
Build speed Fast (goal) Slow Fast Fast Slow Fast
Static binary Yes Yes Yes No (needs BEAM) Yes (with effort) Yes
Learning curve Moderate Steep Low Moderate Steep Moderate
Deployment Static binary, cluster Static binary Static binary Release + BEAM Varies Varies

Strengths by Use Case

Use Case Best Choice Why
Systems programming Rust Zero-cost abstractions, no GC
CLI tools Go, JAPL Fast build, static binary
Web backends JAPL, Elixir/Erlang Supervision, fault tolerance, concurrency
Distributed systems JAPL, Erlang Native distribution, process isolation
Data processing JAPL, Haskell Pure functions, composition, pattern matching
Compiler/PL research Haskell, OCaml Rich type systems, algebraic approach
Concurrent services JAPL, Erlang, Go Lightweight concurrency primitives
Resource-constrained Rust No runtime, no GC, predictable memory

Where JAPL Occupies a Unique Position

JAPL fills a gap in the design space:

  • Erlang's fault tolerance + Rust's resource safety + Go's simplicity + Haskell's type discipline
  • No existing language combines all four: Erlang lacks types and resource safety, Rust lacks lightweight processes and supervision, Go lacks algebraic types and effect tracking, Haskell lacks practical distribution and runtime simplicity

The closest neighbors are:

  • Gleam (typed Erlang, but no ownership, no effect system)
  • Pony (actors + reference capabilities, but not process-isolated, smaller ecosystem focus)
  • Elixir (Erlang with better syntax, but dynamically typed)

JAPL's thesis is that these four concerns -- purity, concurrency, resource safety, and distribution -- are not independent features to be bolted on, but interconnected design decisions that reinforce each other when designed together from the start.