Many recurring text-processing jobs are easy to describe in one sentence yet painful to implement with rules — and routing every input to a remote large model means recurring cost, latency, and dependency on a provider. On September 3, a University of Waterloo team — Yuntian Deng, Pengyu Nie, and Stuart Shieber — published compile by training (arXiv:2609.04199), accepted as an EMNLP 2026 System Demonstration. It compiles a natural-language specification into a local neural function that no longer depends on any teacher model and can be stored, versioned, and composed like ordinary software.
One teacher visit at compile time, zero at run time
The core move is pushing training into the compile step: at compile time, teacher models synthesize task-specific examples, which are used to finetune a small adapter on a compact interpreter. The compiled artifact runs without the teachers — the authors describe a shared 0.6B local interpreter whose compiled functions can be called like ordinary Python functions. The framing question from the authors is blunt: instead of having LLMs solve the same task over and over, why not have them build reusable tools?
This is a follow-up to the same team's July Program-as-Weights (PAW) line: the original compiler produces a program in a single forward pass within seconds; this work adds a higher-accuracy mode where roughly a minute of finetuning buys a large lift on the hard subset.
Where 83.6% comes from
The number needs its context: FuzzyBench-Hard is the subset on which the fast PAW compiler produced no exact matches, and compile by training reaches 83.6% semantic accuracy there. The trade-off is compile time: about a minute rather than seconds. The compiler is deployed in a public interactive service, with demos including a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator, plus a PII masking function that runs locally. The paper and code (github.com/programasweights/compile-by-training) are open, and the paper reached #1 on Hugging Face Daily Papers on its first day.
So what: real engineering value, but you own the long tail
For agent builders, the appeal is not the benchmark but the software-engineering properties: functions can be stored, versioned, and composed, with zero per-call model bills and zero network latency. The community response quickly added sober notes: because training examples are synthesized by the teacher, the compiled function "only knows what the teacher thought to show it" — edge cases the teacher never generated will fail silently. Before trusting it in production, commenters want degradation curves on out-of-distribution inputs, and recompilation from a changed spec needs proof that it converges to the new behavior rather than drifting unnoticed. Turning fuzzy requirements into frozen local functions now has a workable toolchain; the ceiling — a compiled function is never smarter than its teacher — is the part every team hoping to replace live LLM calls should think through first.