How Turf Works
Turf's compiler pipeline is a standard deterministic front-to-back — Lexer → Parser → Resolver →
TypeChecker → Codegen, LLVM backend. Nothing about compilation itself is probabilistic. It lexes and
parses your .tr file into an AST, resolves every name reference, infers and checks every
type, then walks the type-annotated AST to emit LLVM IR — which LLVM's own established optimization
passes turn into a native executable. Full detail: Compiler
Architecture and Compiler Internals.
On top of that deterministic pipeline sits --smart, an opt-in diagnostics layer that runs
only when an error actually fires — no cost, no model inference, on a clean compile. Rather than
pattern-matching formatted error strings, it reads the same structural context (declaration sites,
candidate overloads, related locations) the compiler's own diagnostic engine already attaches to every
error, feeds it to a small locally-run language model via llama.cpp under a
grammar-constrained schema, computes a confidence score from measurable properties of the response, and
only surfaces a suggested fix after speculatively re-parsing and re-validating it against a sandboxed
copy of the compiler state. See The --smart Pipeline and
SLM Integration for the full six-step breakdown.
Compilation pipeline stages
- Lexer: Tokenizes the input source code. Builtin/stdlib names are ordinary identifiers, not lexer-level keywords — the lexer has zero knowledge of the standard library.
- Parser: Recursive-descent, builds an AST of roughly 42 concrete node kinds.
- Resolver: Two internal passes — hoist every top-level name, then resolve every reference against it — so forward references (a function calling one declared later in the file) just work.
- TypeChecker: A visitor-pattern pass inferring and checking a type for every node, including resolving generic type parameters back to their concrete instantiation type.
- Codegen: Walks the type-annotated AST, emitting LLVM IR only after both prior passes have succeeded cleanly — Codegen trusts resolved types and call targets with no re-validation.
Independently of this, a control-flow graph is built for every function and a fixed set of dataflow analyses runs on every compile — definite assignment, liveness, reachability, infinite-loop/recursion detection, and use-after-free/double-free tracking. See Tooling for the full list.
Forward function calls
The compiler uses a pre-pass that hoists function prototypes and fully codegens struct layouts before the
real, authoritative pass runs — this is what lets main() call a helper()
defined later in the same file.
Debug: emit LLVM IR
turf example.tr -o example --emit-llvm