Some code runs while the program compiles rather than when it runs: every macro and every #comptime(...). Its result is put in the program in its place - source for a macro to be compiled like the rest, a constant for anything else. See Compile-time Code and Macros for writing it.
The code that does it is one engine, in bootstrapping/comptime - not a numbered step of its own, since it's needed in the middle of the macro rounds, and by macros and other compile-time code alike. Each round of macros calls it with the program in the intermediate language and a file's macro uses; it gives what each macro gave back - Refingere source, tokens, or any other value - and what the file's other compile-time code worked out. --comptime prints it. Every kind of macro runs, and all the other compile-time code; what can't run while compiling is listed at the end.
What a macro gives back is code, and code can use macros, and #comptime - so running the program's macros once isn't enough. They're run in rounds: each round finds every macro use still in the program, runs them, and takes what they gave through the macro stages into its place. Then the next round looks again, and the rounds go on until one leaves nothing new to run. A macro that gives back a use of itself would go on for ever, so after 64 rounds it's an error.
Derive, attribute and program macros run in rounds of their own first, right after analysis - the phase they run at. What they add, a method or a const, is what the rest of the program then uses, so until they've run a use of it is an unknown name: what analysis found is put aside, and the program is checked and lowered as it is, just to run them. Each runs once; what it gave is parsed as definitions and added to its file - an attribute macro's in place of the definition it's on - and the rounds go on while there's one left to run. Then the program is analyzed again, with it all there, and goes on as usual.
The engine is an interpreter for the intermediate language - the same program the backends will generate code from. That's why lowering comes before macros are expanded: code run while compiling and the same code compiled do exactly the same thing, since they're one program. And the intermediate language is small - a dozen statements, a few dozen expressions, every name a numbered local or a qualified definition - so the interpreter is much smaller than one for the whole language would be.
A value of a builtin type is a real value of the running compiler - a String is a String, an Array an Array - so the builtin types' members work on it as they are, and a macro's result is already a value the compiler can use. What only the program being compiled has is a record of the interpreter's own: one of its structs (its type, by qualified name, and its fields), one of its enum cases, a closure, a reference to one of its functions. An error the program makes or is given - error "...", a builtin that failed - is held as a value too, the way the program holds an error until it's handled, so !!, !?, !> and case Error(e) work on it.
Each call gets a frame: a box for each of the function's locals, its parameters first. Arguments are bound to parameters in order or by name, the rest go into a ..T parameter, and a parameter left out holds a marker until its default is set. A closure keeps the frame it was made in; a call of it gets a frame of its own that shares the boxes of the locals it captures - so it sees what's set outside it, and what it sets is seen outside, while its own parameters and locals are its own, and it can call itself.
Running a statement gives back how it ended: normally, or leaving by a break or continue (with its label), a return (with its value) or a fail (with its error). A block stops at a break for it; a loop goes round on a continue for it and stops on a break; a defer runs its cleanup however its body ended; an inject sets its value for its body, and takes it away after. Anything else goes on up to the function, whose result it becomes. A run that takes more than ten million steps is stopped - a loop that never ends would otherwise hang the compiler.
An operator on the compiler's own values is the compiler's own - a + b on two numbers is its +, on two strings its ++ - once the operands are checked: given values it can't work on, adding a number to a list, it fails the code that did it rather than the compiler. A builtin method or type function is called by name through a table of every one core.md lists, for each number of arguments it can take - a function's rest spread into its ..items - in comptime/methods.rfn, generated by bun run bootstrap:core with core.rfn. A method that takes a function - map, filter - is given the program's closure wrapped, each call run by the interpreter; one that gives an Iterator gives a list, since the interpreter goes through lists. range, #builtin.forever, #comptime.program() and #comptime.definitionsWith work, as do &stdout, &stderr, &env and &fs - the compiler's own, so compile-time code always has the file system. Any other builtin context value is an error there. &fs is the compiler's behind a small value of its own, CtFileSystem, so every path a call on it is given is noted in &compileInputs first - what --watch checks for changes along with the sources.
A function with a yield in it gives its items as they're needed - a for loop takes one at a time, take(n) and takeWhile only as many as they need - so an endless one, for n in #builtin.forever(), works as long as only part of it is used. The interpreter can't pause a function part way, so a generator runs again from the start whenever more items are needed, collecting twice as many each time: the items so far come out the same, as long as it only works with what it's given, which code run while compiling should.
Macros and #comptime.program() see the program as syntax trees - Ast.Struct, Ast.Call, Ast.Program, one type for each kind of node core.md's Ast section lists. Each is one of the compiler's own nodes, wrapped (comptime/ast.rfn) with its kind by that name and the file it's written in, and its methods are that section's: fields(), parameters(), body(), left(), ... each part wrapped in turn, children() and descendants(), span(), and node is Ast.Definition tests its kind or group.
toSource() is the text a node was written as. A node made or changed while compiling - withBody, withAttributes, Ast.parse<Ast.Block>(...), Ast.literal(value) - is written out instead, by 2-parse/write.rfn, which writes any syntax tree as source that parses back into the same tree: brackets only where an operator's precedence needs them, a block's statements a line each. A test writes out every one of the compiler's own files and parses them again to check it. typeOf() is the type type checking worked out - for a macro at the Typecheck phase; before it, undefined.
program.find<Ast.Struct>(where) goes through every node of the program, and keeps the ones of that kind where accepts - the type argument reaches the engine because the intermediate language keeps a call's type arguments (types). #comptime.definitionsWith("command") gives each definition carrying the attribute: its name(), kind(), file(), the attribute's arguments() worked out, its ast(), and its value() - the function to call, the const's value or the type.
A macro's first parameter's type picks the form its input arrives in: String is the text as written, Array<Token> the text lexed, and an Ast type - Ast.Expression, Ast.Block, ... - it parsed into that kind of node, which is an error at the use when it isn't one. A derive or attribute macro gets the definition it's on, a program macro the whole program. After it, by name: an attribute macro's own arguments, the definition's other attributes, and the site it's used at, whenever the macro takes them.
A string or body macro's String or node is source, put where the macro was used; tokens are put there as they are; any other value is written as a constant. A derive macro's source, node, or list of them is definitions added to the file it's in, an attribute macro's takes the place of the definition it's on, and a program macro's is a file of its own in its folder - commands.macro.rfn for the macro commands. When a macro fails - an error it didn't handle - it's reported where the macro was used, or where its error says: error "..." with node.span(), or with the node itself, points at that node. Doing something that can't be done while compiling is reported as its own problem.
A macro runs at a phase - @phase(CompilePhase.Typecheck), or without one the earliest its input exists at. Macros on definitions run right after analysis, as the rounds above describe; one at the Typecheck phase runs once the program's been type checked, sees every expression's type, and the program's checked again after it. String and body macros run after type checking either way; their phase decides whether they see types.
Code run while compiling can use the embedded languages, and gets what the backends make (comptime/embedded.rfn): a #JSON document's values - an object a struct of its keys, a placeholder written as JSON where it is; #HTML and #SVG markup, each placeholder escaped for where it's written; #CSS and #VUE text; #SQL a query, its placeholders bound parameters; #CSV its rows; #SHELL its words; #regex a Regex. A query has no database to run on while compiling.
Lowering marks the rest of the compile-time code - each #comptime(...) - with a comptime node, unless it's in a macro already, which runs while compiling anyway. The engine finds each in a file's functions and consts, runs it in a frame for the function it's in whose locals are all unknown - reading one is reported, 'n' isn't known while compiling - and gives its value as the result of a use of #comptime there, written as a constant and put in place like a macro's result.
@js function belongs to the running program, like everything else outside the language#comptime.definitionsWith in ordinary code, which becomes a constant list - that's the backends' to make, since the list holds functionssrc/5-interpret-not-typechecked is a tree-walking interpreter over the syntax tree. It runs the program's string and body macros, and ran whole programs before they were compiled - it's on its way out.
Next: Optimization.