Frontier models write code and summarize papers fluently, yet exact multi-digit arithmetic still trips them. The standard fixes are two: bolt on a tool so the model calls a calculator, or fine-tune on tens of thousands of drill problems. A single-author preprint filed to arXiv on November 19 (https://arxiv.org/abs/2611.26450) by Tanmay Sah takes a third path: with zero training examples and zero runtime hooks, it compiles an exact arithmetic unit directly into pretrained weights, aiming squarely at the paper's own framing that pretrained language models remain unreliable on exact multi-digit arithmetic.

Circuits, not gradients

The method is called zero-gradient circuit grafting, built on three analytical mechanisms. First, a two-layer Mellin-GLU quadrature over bias-free SwiGLU/GeGLU layers turns each token's unknown RMSNorm scaling into translation along a log-spaced gate grid, with more than 250x dynamic range and IEEE 754 error bounds. Second, cross-token scale cancellation runs through signed Chebyshev Indefinite-RoPE self-sink attention and memoryless SiLU-antisymmetric GatedDeltaNet shifts. Third, an axis-aligned width extension grows the residual stream from d to d+K, paired with sqrt(d/(d+K)) norm compensation, Base-1000 canonical BPE emission, and differential (+254,-256) tied unembeddings. A five-stage automated compiler synthesizes 4-8 digit signed add/subtract/multiply circuits at constant TC^0 depth, plus 3-literal Boolean circuits (N=1,728) and 5-bit Majority/Parity, with a 42-layer Gemma 4 E4B architectural audit. Per the paper's own accounting, the graft touches 1.07M-1.33M active non-zero weights on IBM Granite 4.2 3B (at most 0.036% of the model) and 2.25M-2.39M on Alibaba Qwen 3.5 4B (at most 0.057%).

The self-reported scorecard

On 6-digit arithmetic (N=1,200, BF16), Granite reaches 99.17% and Qwen 100.00% ALU-prefix accuracy, with 100% noted under an FP32 sidecar, and 98%-100% mid-sentence resumption. The same-backbone LoRA baseline (12.8K examples, about 6M added parameters, 400 AdamW steps) manages only 42.2% and 61.7%, with 0% fluent inline continuation. On the memory side, a factorized block-sparse sidecar holds A100 usage at 7.89/10.65 GiB, or 4.60/8.14 GiB under INT8/INT4 plus an FP32 sidecar; the paper reports 42%-44% VRAM savings and 1.96x-2.67x lower 2K-prefill latency. Retention was tested against a pre-registered protocol: all 14/14 Tango splits beyond 2K tokens pass (N=4,392); out-of-domain probing shows 0/11,134 triggers and 0/10,534 activations, with next-token KL shift below 0.002 nats. A zero-token, purely weight-analytic calibration still retains 96.3%/97.0% of arithmetic performance. The paper also states that the compiler, both weight patches, and the frozen pre-registration protocol ship with the accompanying release codebase.

The cold water

First, this is a single-author v1 preprint with no venue listed; every number is self-reported with no third-party replication. Second, the cost is real: per the paper's own ablation, an early one-digit emission variant (K=1,043) crushed GSM8K's first 60 problems from 68.33% to 36.67%, and 18 of the 19 regressions had actually computed exact ALU outputs but were misread by Granite's BPE tokenizer; switching to Base-1000 canonical encoding (K=1,140) restored performance to 70%-76.67%. This echoes a root cause documented independently: digit tokenization, systematically analyzed by Singh and Strouse (https://arxiv.org/abs/2402.14903). Third, the circuits cover only 4-8 digit fixed-width inputs; the paper states plainly that unbounded-length inputs and halting loops exceed single-pass TC^0, and that floats, commas, and code syntax trigger abstention rather than computation. The appendix sorts the 14 failures into three buckets: 8 from base-model digit continuation (7 of them Qwen), 5 from Granite BF16 carry rounding, and 1 parser abstention without a BOS token.

So what

The signal here is not the 100% figure but the route: weights are becoming a programmable substrate. An eight-author study on weight-space programming (https://arxiv.org/abs/2611.09382) by Hu et al., spanning 16 editing trajectories across seven benchmarks and two scales (Qwen3.5-0.8B and 9B), already showed that highly localized weight interventions yield substantial gains, and this preprint shows even gradients can be skipped. The flip side is a warning: a circuit you can weld in, someone else can too - the paper itself cites the compiled-network backdoor ImpNet as precedent. When the next second-hand model lands in your stack, can you tell whether someone else's CPU is already inside?