[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-gated-deltanet-nvfp4-full-4bit":3,"news-related-6d60f9ba-4866-4520-9f12-d955e37f8472":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"6d60f9ba-4866-4520-9f12-d955e37f8472","Gated DeltaNet 全压 4-bit 没掉点:一篇论文拆掉混合 LLM 的量化禁忌","Minima AI 把 Qwen3.8-27B 的 496 个线性层全部压进 NVFP4 4-bit,连公认脆弱的 Gated DeltaNet 门控也没放过。论文测试里精度与 BF16 差距落在种子噪声内,体积缩到 17.5 GiB,prefill 反而快 14-19%,并给出循环层耐量化的机制解释。","混合架构 LLM 的量化圈有一条不成文的规矩:注意力层可以压到 4-bit,循环层不行。9 月 3 日提交到 arXiv 的一篇论文(2609.04098)把这条规矩正面拆了——Qwen3.8-27B 全部 496 个线性层压进 NVFP4 W4A4,连 Gated DeltaNet 的衰减门和写入强度门也没放过,论文测试口径下精度与 BF16 的差距落在种子噪声以内。\n\n## 禁区是怎么来的\n\nQwen3.8-27B 是典型的混合架构:48 层 Gated DeltaNet(GDN)线性注意力,加上 16 层 softmax 注意力。GDN 的循环状态以固定尺寸概括上下文,社区一直担心递归结构里的量化误差会随长度累积。所以 unsloth、RadixArk 的早期 4-bit 版本都把 GDN 块留在 8-bit 甚至 16-bit,门控投影更是重点保护对象。\n\nMinima AI 的 Sergii Kozyrev 和 Davyd Maiboroda 决定检验这个直觉。做法很直接:纯 PTQ 校准,不用 QAT 也不用蒸馏,496 个线性层一刀切全 4-bit。四份模型在完全相同的 serving 环境下对比——vLLM、TP=1、单张 96GB Blackwell:BF16 基线、他们的全 4-bit 版,以及两份保护 GDN 的社区版。\n\n结果:4K\u002F32K 困惑度、MMLU-Pro、GSM8K、AIME'25、GPQA-Diamond、LiveCodeBench、64K 以内 RULER 检索,五任务平均 -0.52;全 4-bit 版反而是四份配方里最小的(17.5 GiB,约为 BF16 的三分之一)且 prefill 最快的(+14-19%)。\n\n## 循环层为什么耐压:四点机制\n\n论文最有意思的部分是解释「为什么」,结论大多反直觉。\n\n**门控恰恰是最不敏感的部件。** 大家最不敢动的衰减门和写入强度门,softplus\u002Fexponential 和 sigmoid 参数化把约 11% 的 GEMM 误差压缩到约 2% 的输出误差——非线性自己当了减震器。\n\n**噪声不会在循环状态里累积。** delta 规则的每次写入都沿着当前 key 方向覆写状态,注入的误差在 32K token 上保持平坦平台,一次状态冲击几百步内就被遗忘;32K 困惑度差距甚至随位置缩小。\n\n另外两点:NVFP4 的 16 元素块缩放把残差流的极端离群点局部化,平衡了各层角色的激活误差;每个 token 的量化成本随上下文被冲淡,而不是复合放大。\n\n## 工程师真正该带走的两条\n\n一是 serving 陷阱:把 GDN 投影融合成单个 GEMM 的 kernel,会静默错配逐模块校准的 NVFP4 checkpoint 的全局 scale。论文修复了这一错配,放出的 checkpoint 预先调和过,开箱即对。二是校准过的 FP8 KV-cache scale 无性能代价,跟着一起发。一句话配方:全部量化,scale 随 checkpoint 一起发。\n\n这份 17.5 GiB 的 checkpoint 已在 Hugging Face 开放(minima-ai\u002Fmnma_qwen3.8_27b_nvfp4),论文提交次日冲进 HF Daily Papers 榜前列(65 个 upvote)。参考:arxiv.org\u002Fabs\u002F2609.04098\n\n对本地部署玩家,这意味着一张 96GB 单卡跑 27B 混合模型,不再需要在精度和体积之间二选一——当然,以上全部数字是论文方自己的测试口径,等第三方复现再下最终结论。更普遍的启示是:混合架构普及之后,「哪些层脆弱」不能再靠直觉投票,得一篇一篇论文去实测。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.04098","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"32fe31e6-9490-48fe-be90-58c184ae8667","en","Hybrid 27B Fully 4-bit: Gated DeltaNet Gates Prove Least Sensitive","All 496 layers of Qwen3.8-27B go NVFP4 4-bit, GDN gates included: matches BF16 within seed noise, 17.5 GiB, 14-19% faster prefill.","Hybrid LLM quantization has an unwritten rule: attention layers can go to 4-bit, recurrent layers cannot. A paper submitted to arXiv on Sep 3 (2609.04098) tests that rule head-on: all 496 linear layers of Qwen3.8-27B pushed into NVFP4 W4A4, including the decay and write-strength gates of Gated DeltaNet, with accuracy landing within seed noise of BF16 under the paper's own benchmarks.\n\n## Where the taboo came from\n\nQwen3.8-27B pairs 48 Gated DeltaNet (GDN) linear-attention layers with 16 softmax-attention layers. Because GDN's recurrent state summarizes the context in a fixed size, the community worried that quantization error inside a recurrence would accumulate over long contexts. Early 4-bit releases from unsloth and RadixArk therefore kept the GDN block at 8- or even 16-bit, with the gate projections as the most protected parts.\n\nSergii Kozyrev and Davyd Maiboroda of Minima AI decided to test the intuition directly: calibration-only PTQ, no QAT, no distillation, all 496 linear layers cut to 4-bit in one pass. Four models ran head-to-head under one identical serving regime — vLLM, TP=1, a single 96 GB Blackwell GPU: the BF16 baseline, their full 4-bit build, and the two community versions that protect the GDN block.\n\nThe outcome: across perplexity at 4K\u002F32K, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench and RULER retrieval to 64K, the five-task average lands at -0.52; the fully quantized build is the smallest of the four recipes (17.5 GiB, roughly a third of BF16) and the fastest at prefill (+14-19%).\n\n## Why the recurrent half survives: four mechanisms\n\nThe most interesting part of the paper is the explanation, and most of it runs against intuition.\n\nThe gates are the least sensitive components of all. The decay and write-strength projections everyone feared turn out to compress roughly 11% GEMM error down to about 2% output error through their softplus\u002Fexponential and sigmoid parameterizations — the nonlinearities act as shock absorbers.\n\nNoise does not accumulate in the recurrent state. Each delta-rule write overwrites the state along the current key direction, so an injected error holds at a flat plateau over 32K tokens and a state impulse is forgotten within hundreds of steps; the 32K perplexity gap actually shrinks with position.\n\nTwo more: NVFP4's 16-element block scaling localizes the residual stream's extreme outliers, equalizing activation error across layer roles; and per-token quantization cost washes out with context instead of compounding.\n\n## What engineers should actually take away\n\nFirst, a serving trap: kernels that fuse GDN projections into a single GEMM silently mis-scale the global scales of per-module-calibrated NVFP4 checkpoints. The paper documents the repair, and the released checkpoint ships pre-harmonized. Second, calibrated FP8 KV-cache scales are performance-free — ship them along. The one-line recipe: quantize everything, ship the KV scales.\n\nThe 17.5 GiB checkpoint is open on Hugging Face (minima-ai\u002Fmnma_qwen3.8_27b_nvfp4), and the paper reached the upper ranks of HF Daily Papers the day after submission (65 upvotes). Reference: arxiv.org\u002Fabs\u002F2609.04098\n\nFor local-deployment users this means a single 96 GB card runs a 27B hybrid without choosing between size and accuracy — with the caveat that every number above comes from the authors' own test setup, awaiting third-party replication. The broader lesson: as hybrid architectures spread, which layers are fragile is a question for measurement, not intuition.","gated-deltanet-nvfp4-full-4bit","2026-09-04T15:08:02Z","2026-09-04T15:08:31.427433Z","2026-09-04T15:08:31.427445Z",true,"agent",28,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"48e1c261-a40a-4c71-9cba-450a459e6ad3","4-bit 模型反超全精度:QAH 把量化从性能税变成第二次蒸馏","quantization-aware-healing-hypernova-60b","2026-08-25T17:20:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"b2d34fba-93ee-469b-9f64-6a7568f89745","Liquid AI 用量化感知蒸馏,把 LFM2.5 4-bit 精度拉回 97%","lfm25-qad-quantization-aware-distillation-edge","2026-08-20T11:00:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"c197245e-0a8a-4028-9ad2-6547bdc01be5","BitNet 团队把 1.58-bit 量化推进到 embedding:检索向量也可以\"训练时就压\"","bitnet-1-58bit-embedding","2026-07-18T03:00:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"93981299-c75e-4140-ad2c-ae68fb5d27ce","CAT-Q：512 样本把 235B LLM 压到 1.58-bit，成本降 10 万倍","cat-q-1-58-bit-512-samples-icml-oral","2026-06-26T20:25:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"199cd4ef-f092-45a5-8635-91778dd2bce2","编译即训练：一句规约炼出 83.6% 准确率的本地神经函数，教师模型只用一次","compile-by-training-neural-functions","2026-09-04T23:08:03+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"7a6d28b6-65da-4a29-96b1-dedb9894de97","随机驱逐追平最强打分器:Salesforce 重写 KV Cache 压缩常识","random-attention-kv-cache-eviction","2026-09-04T19:08:26+00:00"]