Hybrid LLM quantization has an unwritten rule: attention layers can go to 4-bit, recurrent layers cannot. A paper submitted to arXiv on Sep 3 (2609.04098) tests that rule head-on: all 496 linear layers of Qwen3.8-27B pushed into NVFP4 W4A4, including the decay and write-strength gates of Gated DeltaNet, with accuracy landing within seed noise of BF16 under the paper's own benchmarks.
Where the taboo came from
Qwen3.8-27B pairs 48 Gated DeltaNet (GDN) linear-attention layers with 16 softmax-attention layers. Because GDN's recurrent state summarizes the context in a fixed size, the community worried that quantization error inside a recurrence would accumulate over long contexts. Early 4-bit releases from unsloth and RadixArk therefore kept the GDN block at 8- or even 16-bit, with the gate projections as the most protected parts.
Sergii Kozyrev and Davyd Maiboroda of Minima AI decided to test the intuition directly: calibration-only PTQ, no QAT, no distillation, all 496 linear layers cut to 4-bit in one pass. Four models ran head-to-head under one identical serving regime — vLLM, TP=1, a single 96 GB Blackwell GPU: the BF16 baseline, their full 4-bit build, and the two community versions that protect the GDN block.
The outcome: across perplexity at 4K/32K, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench and RULER retrieval to 64K, the five-task average lands at -0.52; the fully quantized build is the smallest of the four recipes (17.5 GiB, roughly a third of BF16) and the fastest at prefill (+14-19%).
Why the recurrent half survives: four mechanisms
The most interesting part of the paper is the explanation, and most of it runs against intuition.
The gates are the least sensitive components of all. The decay and write-strength projections everyone feared turn out to compress roughly 11% GEMM error down to about 2% output error through their softplus/exponential and sigmoid parameterizations — the nonlinearities act as shock absorbers.
Noise does not accumulate in the recurrent state. Each delta-rule write overwrites the state along the current key direction, so an injected error holds at a flat plateau over 32K tokens and a state impulse is forgotten within hundreds of steps; the 32K perplexity gap actually shrinks with position.
Two more: NVFP4's 16-element block scaling localizes the residual stream's extreme outliers, equalizing activation error across layer roles; and per-token quantization cost washes out with context instead of compounding.
What engineers should actually take away
First, a serving trap: kernels that fuse GDN projections into a single GEMM silently mis-scale the global scales of per-module-calibrated NVFP4 checkpoints. The paper documents the repair, and the released checkpoint ships pre-harmonized. Second, calibrated FP8 KV-cache scales are performance-free — ship them along. The one-line recipe: quantize everything, ship the KV scales.
The 17.5 GiB checkpoint is open on Hugging Face (minima-ai/mnma_qwen3.8_27b_nvfp4), and the paper reached the upper ranks of HF Daily Papers the day after submission (65 upvotes). Reference: arxiv.org/abs/2609.04098
For local-deployment users this means a single 96 GB card runs a 27B hybrid without choosing between size and accuracy — with the caveat that every number above comes from the authors' own test setup, awaiting third-party replication. The broader lesson: as hybrid architectures spread, which layers are fragile is a question for measurement, not intuition.