Linear-attention models have been sold on one promise: replace the Transformer's key-value cache with a fixed-size recurrent state. Context length no longer inflates memory, but every concurrently decoded sequence still holds one copy — so shrinking the state directly buys batch size on a given accelerator. The catch is that this state is read and rewritten at every step. Rounding it to low precision after each generated token lets error compound, and existing state quantizers report a grim verdict: four bits lose accuracy unless you raise the width or fine-tune. PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States (Haller, Laitenberger, Akbik; submitted October 21) argues the opposite: four bits can hold without retraining.

First lever: stop rounding every step

PACE is a post-hoc quantizer. Borrowing from chunked kernels — prefill already processes the state once per chunk — it stretches that interval into decoding: round the state every eight generated tokens, park the in-between tokens in a small decode buffer, and at each flush advance the buffer into the state before requantizing.

The numbers show the effect size. On Qwen3.5-9B, the full-precision state averages 91.0 across GSM8K, IFEval, and RULER-32k. Compressed to four bits at 4.60 resident bits per value and rounded every token, that drops to 78.1. Rounding every eight tokens (5.28 resident bits) recovers 82.9; every thirty-two tokens (7.64 resident bits) reaches 85.6. Qwen3.5-4B traces the same curve: 88.7, 76.7, 84.1, 86.0.

Most of the INT4 loss, in other words, comes from how often you round, not from the four-bit width itself.

Second lever: large entries gather into columns

The paper also reports a structural observation: in the gated-delta state, large entries cluster into columns, because a few channels of the delta value written at each token stay large across many tokens. The authors confirm it by permuting those channels every token — the large entries then stop forming columns. If they cluster by column, the quantization scale should be shared within a column, along the key axis rather than the value axis.

The ablation is clean. At the same 5.28 resident bits with g=32 groups, value-axis grouping gives a WikiText-103 perplexity of 8.158 while key-axis grouping gives 7.719 — against 7.713 for the full-precision state. Q-Mamba's DSQ sits at 8.320 in the same table. Adding head-level precision allocation takes the 4B perplexity to 7.687 and IFEval to 81.52, above DAMP INT4's 79.48.

Measured comparison and its costs

Evaluation covers four hybrid models from 4B to 35B parameters — Qwen3.5-4B/9B/27B and Qwen3.6-35B-A3B — with every configuration except DAMP INT4 rounding every eight tokens. The authors report that PACE-4/8, with mixed four- and eight-bit heads, is competitive with eight-bit DAMP on the three larger backbones at about two thirds of its resident memory, and that fully four-bit PACE-4 scores higher than Q-Mamba DSQ on every backbone at equal resident bits. For speed, they time TPOT on a single H100 PCIe GPU at a 512-token context across batch sizes 1 to 128: periodic INT4 decoding is cheaper than per-token bf16 decoding from batch 16 upward, and the gap widens with batch size.

The limits are stated too: head decays, column structure, and head ranking are measured only on these four Qwen hybrid models; the prefill rounding period is fixed at the kernel's 256-token chunk; and throughput is timed only for the uniform four-bit state.

So what

The value here is not "four bits got a bit more workable." It is a different framing of the problem: the dominant source of quantization loss is rounding frequency, not the bit width itself — a conclusion that applies to every recurrent-state architecture, including the chunked forms of gated linear attention, Mamba-2, and DeltaNet. For teams running hybrid linear-attention models in long-context serving, that is a throughput path requiring no model changes, no retraining, and only a different decoding schedule.