[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-quantization-aware-healing-hypernova-60b":3,"news-related-48e1c261-a40a-4c71-9cba-450a459e6ad3":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"48e1c261-a40a-4c71-9cba-450a459e6ad3","4-bit 模型反超全精度:QAH 把量化从性能税变成第二次蒸馏","Multiverse Computing 提出 Quantization-Aware Healing:把 GPT-OSS 120B 压到 60B 再量化 MXFP4 后,不从恢复版而从原始模型蒸馏修复,9 项基准中 7 项反超自家 bfloat16 版本,权重内存约为四分之一。","把大模型做小的标准流水线是两步:先做结构压缩,砍掉层数或注意力头;再把权重量化到 4-bit。两步都在省钱,也都在损伤推理、数学、代码能力,所以部署流水线会在上线前加一步「修复」(healing)。gpt-oss、NVIDIA Nemotron、Multiverse Computing 的 Hypernova 60B,走的都是这条路线。\n\n修复该怎么做,一直没被认真回答。Multiverse Computing 8 月 21 日挂上 arXiv 的论文(arXiv:2608.20953)给出答案,还附赠一个反直觉结果:GPT-OSS 120B 压到 60B、再量化 MXFP4 后,修复出的 4-bit 模型在 9 项基准里 7 项赢过自己的 bfloat16 版本——更小、更便宜,还更准。\n\n## 旧方法的两个坑\n\n主流方案 QAT 在前向里插入伪量化算子继续微调,论文实测它收敛慢,训过头还会崩,要人工 early stopping 兜底。另一方案 QAD(量化感知蒸馏,Liquid AI 上周公开过同类用法)冻结全精度 teacher 蒸馏,但有隐藏前提:存在「真正全精度」的同架构模型。结构压缩过的模型从未被独立全精度训练过,唯一 teacher 是恢复出的 bfloat16 checkpoint——本身就是蒸馏近似,拿它教 student 等于把上限锚在降级目标上。\n\n## QAH:换个 teacher,问题就解了\n\nQAH 只改一处:蒸馏回到压缩前的原始模型。Teacher 全尺寸全精度,student 一半参数跑 MXFP4,架构不必相同——teacher 的输出分布与架构无关,KL 散度照样对齐 logits。换个角度:量化不再是修复后的有损后处理,而是对着原始 teacher 再做一轮完整蒸馏,补上 bfloat16 版本没来得及学的东西。长上下文复用配套论文的分块 KL 损失,32k token 语料能塞进固定显存。\n\n## 数字说话\n\n反超项里涨幅最大的是压缩最伤的能力:长上下文推理 AA-LCR 35.3→42.7(+7.4),数学 AIME 2025 70.7→76.3(+5.6),Aider +2.7、工具调用 τ²-bench +2.3、GPQA Diamond +1.7。输的两项 MMLU-Pro(−0.2)、SciCode(−1.4)都在 1.5 分内。对 120B teacher 本尊:LiveCodeBench 拿 66.5,反超其 66.0。\n\n与 QAT 的对照消融(9B 量化到 MXFP4):峰值相当(54.9 对 54.6),但 QAH 约 100 步到峰、QAT 要约 700 步,慢约 7 倍;QAT 过峰后到 1200 步累计掉近 19 分,QAH 稳定不漂移——KL 蒸馏把 student 绑在固定 teacher 分布上,追平后没有漂移压力。\n\n成本侧:4-bit 权重内存约为 bfloat16 四分之一,参数是 teacher 一半,每 token 计算约减半。成品已以 Hypernova-60B 开放权重发布。\n\n## 量化不是税,是机会\n\n最值得记住的是叙事翻转:量化从「为效率交的精度税」变成「一次额外的教学机会」。两个落地细节:作者在社区问答确认 QAH 不需要原始训练语料,公开数据加任务数据就够;方法明确瞄准「不需要数周超参搜索」的可复用配方。\n\n下次有人断言 4-bit 必然掉点,把这篇甩给他:在正确的蒸馏目标下,更小的模型可以同时更便宜和更准。\n\n参考:arXiv:2608.20953,官方博客 huggingface.co\u002Fblog\u002FMultiverseComputingCAI\u002Fquantization-aware-healing。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.20953","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"8b695fda-5001-40a9-b527-450c75f6546a","en","QAH: 4-Bit LLM Beats Its Own BF16 Source on 7 of 9 Benchmarks","Multiverse's QAH distills 4-bit students from the original uncompressed model; a 60B MXFP4 model beats its bfloat16 source on 7 of 9 benchmarks.","Shrinking a large language model usually means two steps: structural compression first, cutting layers or attention heads to reduce the parameter count, then quantizing the remaining weights down to 4 bits to shrink memory and compute further. Both steps save real money, and both quietly erode reasoning, mathematics, and code generation — the capabilities people care about most. That is why serious deployment pipelines add a recovery stage, usually called healing, before production. Recent open-weight releases such as gpt-oss, NVIDIA's Nemotron family, and Multiverse Computing's own Hypernova 60B all rely on some version of this compress-then-heal approach.\n\nHow that healing stage should actually be done has been an open question. A paper published on arXiv on August 21 by Multiverse Computing, \"Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs\" (arXiv:2608.20953), offers an answer with a counterintuitive result: on a pipeline that compresses GPT-OSS 120B down to 60B parameters and then quantizes to MXFP4, the healed 4-bit model beats its own bfloat16 full-precision version on 7 of 9 benchmarks — smaller, cheaper, and more accurate at the same time.\n\n## Where the old recipes fall short\n\nThe dominant recovery recipe is quantization-aware training (QAT): it inserts fake-quantization operators into the forward pass and keeps fine-tuning on a task loss. In the authors' measurements it converges slowly and collapses once training continues past its peak, forcing careful hand-tuned early stopping. The alternative, quantization-aware distillation (QAD) — the approach Liquid AI has publicly used — freezes a full-precision teacher and distills directly into the quantized student through a KL-divergence loss. But QAD carries a hidden assumption: that a genuine full-precision version of the exact same architecture exists to serve as teacher. A structurally compressed model was never independently trained at full precision, so the only candidate teacher is the recovered bfloat16 checkpoint — itself a distilled approximation. Distilling from it anchors the student to a degraded target and caps accuracy at that checkpoint's own ceiling.\n\n## QAH: change the teacher, and the problem dissolves\n\nQuantization-Aware Healing (QAH) makes exactly one change: supervision comes from the original, pre-compression model instead of the recovered one. The teacher is full-size and full-precision; the student is half the parameters and running in MXFP4. The two do not even share an architecture — a teacher's output distribution is architecture-agnostic, so KL divergence on the logits still transfers. Reframed this way, quantization is no longer a lossy post-processing step after healing; it is a second, full pass of distillation against the original teacher, picking up information the earlier recovery stage never had time or data to transfer. For long contexts, the team reuses the chunked KL-divergence loss from its companion paper on efficient distillation, computing the loss one slice at a time so that healing corpora with documents up to 32k tokens fit inside a fixed GPU memory budget.\n\n## The numbers\n\nThe largest gains land exactly where compression hurts most: long-context reasoning AA-LCR rises from 35.3 to 42.7 (+7.4), math AIME 2025 from 70.7 to 76.3 (+5.6), agentic coding Aider +2.7, tool use τ²-bench +2.3, GPQA Diamond +1.7. The two losses, MMLU-Pro (−0.2) and SciCode (−1.4), are each under a point and a half. Against the 120B teacher itself, the 60B QAH model scores 66.5 on LiveCodeBench, beating the teacher's 66.0.\n\nThe head-to-head ablation against QAT (a 9B model quantized to MXFP4) explains the mechanism: both reach a similar peak (54.9 for QAH vs 54.6 for QAT), but QAH peaks in about 100 steps while QAT needs about 700 — roughly 7 times slower — and QAT then sheds nearly 19 points by step 1,200, while QAH holds steady near its peak. KL distillation ties the student to a fixed teacher distribution, so once it catches up there is no pressure left to drift.\n\nOn the cost side, the 4-bit model uses roughly 4 times less weight memory than bfloat16, half the teacher's parameter count, and roughly half the compute per token. The resulting model is released open-weight as Hypernova-60B.\n\n## Quantization is not a tax — it is an opportunity\n\nThe most memorable part of this paper is not any single score but the narrative flip: quantization stops being a precision tax paid for efficiency and becomes an extra opportunity to teach the model. Two practical details stand out for deployment teams. First, the authors confirmed in the community discussion that QAH does not require the original training corpus — an internal selection of high-quality public data plus task-specific datasets was enough — which matters for third parties who cannot access the original provider's data. Second, the method explicitly targets a recipe deployable without a multi-week hyperparameter search.\n\nThe next time someone insists that 4-bit must cost accuracy, hand them this paper: with the right distillation target, a smaller model can be cheaper and more accurate at once.\n\nReferences: the paper at arXiv:2608.20953 and the official blog post at huggingface.co\u002Fblog\u002FMultiverseComputingCAI\u002Fquantization-aware-healing.","quantization-aware-healing-hypernova-60b","2026-08-25T17:20:00Z","2026-08-25T17:14:29.406630Z","2026-08-25T17:14:29.406638Z",true,"agent",41,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"b2d34fba-93ee-469b-9f64-6a7568f89745","Liquid AI 用量化感知蒸馏,把 LFM2.5 4-bit 精度拉回 97%","lfm25-qad-quantization-aware-distillation-edge","2026-08-20T11:00:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"c197245e-0a8a-4028-9ad2-6547bdc01be5","BitNet 团队把 1.58-bit 量化推进到 embedding:检索向量也可以\"训练时就压\"","bitnet-1-58bit-embedding","2026-07-18T03:00:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"93981299-c75e-4140-ad2c-ae68fb5d27ce","CAT-Q：512 样本把 235B LLM 压到 1.58-bit，成本降 10 万倍","cat-q-1-58-bit-512-samples-icml-oral","2026-06-26T20:25:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"cac485ab-a429-4ebc-88c6-a1f924f978ff","AWS 开源 KeysAndValues:微调时就让模型学会“遗忘”,单张 A100 撑住 128K","aws-keysvalues-sparse-attention-finetuning","2026-08-26T05:20:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"c94bdf86-5de9-49fe-8c98-0f5c47611bfe","SGLang v0.5.18 发布:大模型冷启动提速 2.38 倍,710 个 PR 都改了什么","sglang-v0-5-18-cold-start-2-38x","2026-08-24T23:15:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"988bbfb8-672a-4c6c-98f7-3a170b6bd8b3","Macaw 把 LFM2.5 装进 1.5GB:4-bit 端侧 LLM 跑 Mac 控制工具链","macaw-lfm25-15gb-edge-mac-agent","2026-08-24T06:00:00+00:00"]