A serving stack picks a weight format per model and deployment: int8, fp8, or int4. Evaluating task quality for every combination is slow, so the cheap route is to predict the damage directly. The standard prediction is second order — the KL divergence between a model and its quantized copy is approximately one half times delta transpose F delta, with F the Fisher information of the unquantized model. That expansion is the bedrock under GPTQ, HAQ, and the rest of the Hessian-aware family, and it costs one forward and backward pass per calibration sequence, after which every candidate format is a dot product per tensor. Almost nobody asks the follow-up question: is the number that comes out actually trustworthy?

This paper does something rare. It freezes every prediction, every criterion, and every file checksum before the GPU run, then publishes a ledger recording which frozen predictions held and which did not. The authors are Dezhen Lu from the University of Tübingen and Zixiang Ni from Xi'an Jiaotong University. Five models, three quantization schemes, all on GH200.

Ranking survives, absolute values do not

The good news first: relative predictions genuinely hold. Sweeping the perturbation scale from one eighth to one — a 16-fold range — the ratio R of measured KL to predicted KL stays between 0.99 and 1.05. For int4 group sizes of 32, 64, and 256, calibrating on group size 128 predicts the measured KL within 0.96 to 1.01. Stochastic rounding raised the KL by 2.25 times against a predicted 2.28 times. As a tool for ranking candidate formats, this screen is solid.

The bad news is the absolute number. The per-sequence quadratic form is heavy-tailed: the top 5% of sequences carry up to 24% of the prediction, and a Hill estimate puts the tail index at 1.7 to 2.4 on Qwen3-8B and Qwen3-32B — right at the edge of infinite variance. The same model under three different sampling rules gave plain-average ratios of 0.85 to 0.88, 0.66 to 0.70, and 0.56 to 0.58. That is a thirty-percent swing in the reported value. The fix is a 5% trimmed average, which brings 10 of 12 cells on Qwen3-4B and OLMo-2 within 15% of measured KL.

bf16 backward passes skew the prediction by four times

The paper also reproduces a 4.2 times over-prediction reported in earlier work, running that experiment's own code unchanged. The prediction came out 4.04 to 4.52 times higher than the measured KL. The cause is not the model but the arithmetic of the backward pass. On one sample set, a few sequences have early-layer gradients that blow up element-wise in bf16 — one reaches 1237 to 1582 times its fp32 value — and squaring them drags the mean with it. The error lives in the residual-stream gradient and grows toward the input. Put the lower half of the backward pass in fp32 and the error at the bottom falls to 0.017 of the all-bf16 value; put only the upper half in fp32 and it stays at 0.96.

At 32B, the diagonal form breaks outright

On Qwen3-32B, int8 and fp8 held (trimmed ratios 0.99 and 0.86). int4 did not: measured KL was 5.18 times the prediction with fp32 gradients and 5.17 times with bf16. Per sequence, the diagonal form does not rank int4 at all (Spearman 0.09); the tensor-block form ranks it weakly (0.25). The excess sits in one tensor — the last layer's down projection carries 99% of the gap, and its block form is 32 times its diagonal. Going one level deeper, two of that tensor's 25,600 input columns carry 96% of the column-block form. Stochastic rounding at the same four bits removes the coherence and cuts that tensor's KL by 15 times.

The pre-registered ledger in the paper records every frozen criterion next to its outcome, including the ones that failed.

Mixture-of-experts models behave differently again. The response is not quadratic at all: int8 KL scales as alpha to the 0.63, and the share of expert-set flips grows as alpha to the 0.41, from 6.6% at a quarter scale to 11.7% at full scale. The consequence inverts the dense case — the smaller the perturbation, the worse it predicts. Trimmed, int8 comes out at 4.1 times the prediction at full scale and 27 times at a quarter scale. The paper's pre-registered MoE test was inconclusive because the dense control failed, and it reports that verdict as inconclusive rather than as a result.

Ask about the noise floor before comparing anything

The paper measures a noise floor in the serving engine itself. Whenever the reduction order changes — different kernel selection, two-way or four-way tensor parallelism, batch-invariant versus default kernels — each sequence moves by 0.17 to 0.22 nats. KL measured across engine instances or configurations therefore carries that floor, and differences below roughly 0.2 nats per sequence are not resolvable unless the kernel selection is fixed. Fixing it means batch-invariant kernels, which run at 0.55 times the throughput of the default mode, or 0.81 times in the teacher-forced setting. Thinking Machines Lab's write-up on defeating nondeterminism in LLM inference covers the mechanism in more depth, and an independent study of numerical sources of nondeterminism in LLM inference separately finds that BF16, despite being widely used, exhibits substantial variance compared to FP32's near-perfect determinism.

A screener built from those conclusions

The paper finishes by shipping the screener its own findings imply: one streamed fp32 Fisher pass, 1.88 seconds per sequence on a single GH200 (8.0 minutes for 256 sequences, three formats, 224 tensors), reporting a band of 0.75 to 1.50 rather than a single number, plus a flag for aligned tensors and a per-layer format allocator. On Mistral-7B-v0.3 — a family that took no part in any of the earlier analysis — all six measured cells fall inside the band, with ratios of 1.03 to 1.12, all three KL budgets are met, and vLLM throughput is predicted within 0.4%. The post-hoc comparison matters more than the criterion: simply predicting the nearest uniform format would have been off by 0.7 to 4.5%, and bf16 by 3.7 to 7.3%, against the formula's 0.4%, which is the repeat-to-repeat spread. The whole pre-registered test used 0.77 GPU-hours.

The honesty is the part worth copying

The ledger records the misses too. With only eight calibration sequences, mixed configurations came out at measured-to-predicted ratios of 1.20 to 1.41; at 256 sequences they settled to 1.06 to 1.08. The authors call it an optimizer's curse: the allocator upgrades the layers whose block terms are largest on the calibration sequences, so the layers left at int4 are, by selection, the ones whose terms came out low there. LAMBADA accuracy (5153 items) ranged from 73.2% under int4 to 75.0% under int8, with bf16 at 74.8% and the three budgets keeping 74.5% to 74.7%. The throughput gains are modest at this operating point: int4 serves 17% more tokens per second than bf16, int8 3%, and the three budgets 8%, 5%, and 4%. The ten-percent throughput tolerance did not discriminate here, because all three uniform throughputs already sit within ten percent of the mixed ones — the paper says so outright.

So what

For anyone running quantization in production, the operational reading is clear. Use this screen to rank formats; do not use its raw number as a budget. If you need an absolute figure, trim 5%, compute per-sequence gradients in fp32, and run an alpha scan to confirm the perturbation is still inside the quadratic regime. And check the tensor-block form separately at the 32B scale, because there the diagonal approximation fails outright rather than degrading gradually.

What is genuinely scarce here is not the conclusion but the method: freeze the criteria, keep the checksums, then publish the predictions that did not survive. In measurement work the expensive thing is rarely the result — it is the measurement nobody was willing to reproduce.