Code Generation

The last steps turn the program into something that runs: a WebAssembly module, or a JavaScript module. --wasm and --js ask for one or the other. The TypeScript compiler generates them from the syntax tree. The bootstrapping compiler generates them from the intermediate language, as optimization leaves it, in bootstrapping/11-compile, following the TypeScript backends' conventions and using their runtimes, so a program behaves the same whichever compiler built it. In both, --run runs the result on Bun - in the bootstrapping compiler a JavaScript module unless --wasm asks for WebAssembly, built in the program's hidden .refingere/run folder (or --out) and run as a Command, sharing the terminal, with the arguments after --.

One value model, two targets

Both backends run the same program the same way - the same standard library, the same method dispatch, the same errors - and every test that compiles a program runs it through both, which must print the same. What they share is how values are represented while the program runs: every value carries its type, so an operator or a method call can choose what to do by the values it's given. An F64 is a plain float; a small S64 is held as it is; every other number type is tagged with its own. Type checking is gradual, so what it couldn't work out is checked here, as values meet.

A method call is dispatched on the value: its struct's or enum's own method, then one written for its kind of builtin value, then one written for an interface. Each backend builds a dispatcher per method name for that - which is why the intermediate language keeps call-method as it is, and lists which of the program's functions each method is.

An operator on a struct or an enum is a method call too - = calls equals - except where the answer can't depend on one. Comparing two cases of an enum that has no equals of its own, neither carrying anything, compares which cases they are and calls nothing; and token.kind = TokenKind.Space, against a case written out, tests the case in place, before anything's made - the way a match does. A value of any other type there goes the usual way. Such a case is one value, made once, wherever it's written.

WebAssembly

src/6-compile-wasm generates WebAssembly text (compile.ts) and assembles it into a binary module itself (wat.ts), so no outside toolchain is needed. Each module includes its runtime:

  • runtime.wat - how values are stored, text, lists, equality, number formatting, injection, files, and the garbage collector: a copying collector, run at safe points - a function's start and each loop round - with every live value kept on a shadow stack it can find.
  • typed.wat - every number type's arithmetic, conversions and comparisons.
  • numbers.wat - the number intrinsics, and bigint.wat the arbitrary- precision integers the exact float code is written on.
  • regex.wat - the regular expression engine.

The standard library - the documented members of the builtin types - is written in Refingere, in lib/, and compiled along with the program, only as far as the program reaches it. The module is a WASI program: wasmtime runs it as it is, and Bun's own WebAssembly runs it when wasmtime isn't there.

JavaScript

src/6-compile-js is the WebAssembly backend's twin: the same program structure and standard library, as an ES module running on runtime.ts. JavaScript does what the WebAssembly runtime does by hand: its closures capture variables, its generators run yield and @async, its garbage collector reclaims memory, its RegExp matches. --target=js picks it, and a web page's code uses it.

An integer there - any integer type, S64 too - is tagged with its type, and its value is a JavaScript number while it's a safe integer, anything from -(253 - 1) to 253 - 1, and a bigint only past that. So everyday arithmetic - an index, a count, a position in a file - is a number's, several times quicker than a bigint's, and the results are just the same: a result is checked against its type's own limits either way, and one that leaves the safe integers is worked out again exactly. An integer written in the program is made once, a constant of the module.

From the intermediate language

Generated from the intermediate language, each backend has much less to do than from the syntax tree: no patterns, no loop modifiers, no defaults or format strings - lowering has taken those apart. A statement becomes the matching WebAssembly or JavaScript construct: an if, a block or loop left by label, a defer run on every way out of its block. Locals are already numbered, and a closure lists what it captures.

11-compile/program.rfn is what both backends need to know about the program as a whole: every function, const and type by name, and the methods written for each type, the program's before the standard library's. embedded.rfn plans an embedded language's document once, for both. tables.rfn is the TypeScript backends' tables - number kinds, operators, builtin methods, intrinsics, the Unicode data - generated by bun run bootstrap:core.

  • js/ writes one ES module importing runtime.ts. A local is a JavaScript variable - one a closure captures a cell, { v }, handed to the closure when it's made, so a closure made in one round of a loop keeps that round's. A generator, and an @async body, is a function* run a step at a time; a defer or an inject block's end is a finally. JavaScript is called as the TypeScript backend calls it - a @js declaration's module (the runtime's host: ones among them) - and a @js definition is one of the module's exports; @serve starts the host's server on the program's routes. The library's host:thread declarations, Promise.parallel's, are called with the program's own values, and a thread imports the module itself to run the work: it finds the function by its place in P.generics, the list of every function the module has.
  • wasm/ builds the program's code as trees of the text format's S-expressions (code.rfn: watCall("$box", [value]) is (call $box value)), never as text to read back, and assembles them, with runtime.wat and the rest, itself (assemble.rfn, byte for byte what wat.ts makes). The runtime's files are read once, however many modules a run builds on them. A function's values live in its shadow-stack frame, where the collector sees them. Anything an expression works out that a later part of it could outlive goes there first, since evaluating that later part can collect. A local a closure captures is a cell, shared by reference - a new binding of it a new cell. A generator, and an @async body, is a state machine: every local a cell, its statements in numbered blocks that the next step resumes at, the library's AsyncTask running an async one on to each await; a defer or an inject split into blocks is undone on every way out of them but a yield, and a new binding is a new cell put in the step's environment. A module is plain WASI - unless it calls JavaScript or JavaScript calls it - as an HTTP server does, through the @js function @serve adds: then it's built for the TypeScript backend's JavaScript host (interop.rfn) - each @js declaration an import, each @js definition an export, called with the program's injected values - with the manifest the host loads it with, program.js.json, beside it. A host:thread import gets the module's values as they are: a thread is another instance of the same module, so a function crosses as its table index, and rt_call calls it there. A test module also exports each test on its own - rt_test_count, rt_test_name and rt_test_run - for the host to run them side by side.

--out=<folder> is where the result goes: program.mjs, or program.wasm. Without it, the module goes to standard output - the JavaScript's text, or the WebAssembly's bytes. --tests compiles the program's @test functions to run instead of its @main - those in no stage; --tests=e2e those in the e2e stage too, and --tests-all every one (see Testing) - and --errors-only leaves out the tests that pass. --bench compiles its @bench functions to time instead - the library's lib/bench.rfn runs them - and with --run, once they're timed, compiles @main again as profile.mjs or profile.wasm, runs it under Bun's profiler (bun --cpu-prof-md) and lists the functions it spent its time in, from Bun's own report (see Benchmarking). --library=<folder> is where the standard library and the runtimes are. A JavaScript module runs with bun program.mjs, a WebAssembly one with wasmtime --dir=. program.wasm. The bootstrapping compiler compiles its own tests either way, and they pass as they do when the TypeScript compiler runs them.

Back to The Compiler.