Parsing turns a file's tokens into a syntax tree: the structure the text describes. Where the tokens are a, +, b, *, c, the tree says it's an addition whose right side is a multiplication. Every later step works on the tree.
Given a file's tokens. Gives its syntax tree: the definitions at the top of the file, and its implement blocks. In bootstrapping/2-parse; --ast prints it. The tree's node types are in 2-parse/ast/ - one struct per construct, grouped by the enums that say what can stand where: Expression, Statement, Pattern, MatchPattern, Definition. The shape a macro sees is on Syntax Tree, and the grammar on Syntax.
Only what the program means. Space tokens - whitespace and comments - are dropped before parsing starts, and so are brackets that only group: (a + b) * c is a multiplication whose left side is an addition, with no node for the brackets. Each node records its span - where it starts and ends in the file - so anything found about it later can point at the exact text. Spans are the only link back to the source; line and column are worked out from them when a diagnostic is shown.
Spellings are made one before the tree is built. Every operator that has another way of being written is read as one form: ≔ as :=, ≠ as !=, ∧ as and, = as == - so later steps only ever see one. An empty pair of brackets, one token to the lexer, is split in two, so an empty list is just a list with nothing in it.
The parser reads keywords, brackets and operators by each token's spelling rather than its text, and the tree keeps an operator the same way: a BinaryExpr's operator is Spelling.Plus, not "+", and so are a prefix, postfix or comparison operator, a member access's . or ?., a call's or index's guard (Spelling.None for a plain one) and a set's operator - Spelling.Assign for :=, Spelling.Plus for +=. spellingText gives an operator's text back, for a message or the intermediate language. A macro still sees an operator as its text.
It's a recursive-descent parser: a function per construct - a definition, a statement, an expression, a pattern - each reading the tokens its construct is made of and calling the others for what's inside it. It reads through a Cursor over the tokens, injected for the whole parse as &tokens, so none of those functions has to pass it along.
Operators are parsed by precedence climbing. An expression starts with an operand - a name, a literal, a bracketed expression, a prefix operator and its operand - and then, as long as the next token is an operator that binds tighter than the one the expression is inside, takes it and its right side. Each infix operator has a precedence and a direction, from the table on Operators: ^ is right to left, everything else left to right. Comparisons are a chain - a < b <= c is one node with three operands - and calls, indexes, member access and postfix operators bind tightest of all.
A statement ends with its line. Inside brackets it doesn't - an argument list or a list can go over as many lines as it needs - so the parser counts how deeply it's nested, and only treats a new line as an end at depth zero. Whether two tokens are written right against each other matters too: it's what tells a call from two things side by side.
A type is parsed as an expression, in a mode where < opens type arguments instead of comparing, and ? and ! make optional and error types: Map<String, Array<U64>>?, ParseError ! S64. That way a type can appear anywhere an expression can - passed to a function, or used as a value - and the steps after tell the two apart by what the name means.
A builtin macro's string or body - #format"...", #SQL {{{ ... }}} - is one token, but its #{ expression } placeholders are code, written where the macro is. The parser takes them apart on request (2-parse/placeholders.rfn): the text between them, and each placeholder's expression parsed with its spans where it really is in the file. Analysis, type checking and lowering all ask for them, and get the same nodes each time.
--ast writes the tree as Refingere values - every node with its span. It's long; its use is keeping a file's tree between runs, and looking at one construct's shape.
What the tokens can't be read as: Expected 'then', found '{', pointing at the token that didn't fit - or just past the last one, when a file ends too soon. A file's first parse error stops its parse, since the parser can't know what was meant after it; each file is parsed on its own, so one file's error doesn't hide another's.
src/2-parser is built from parser combinators, one module per construct, with indentation checked as part of the grammar, and operators parsed from each folder's operator table. Its nodes carry each later step's findings on them, where the bootstrapping compiler keeps them beside the tree.
Next: Analysis.