[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-qwen3-2-bit-fp16-planning-loop-rescue":3,"news-related-f33cd4ce-46c8-4ce0-8a8a-090b1359dc34":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"f33cd4ce-46c8-4ce0-8a8a-090b1359dc34","2-bit 量化翻车实录：Qwen3 推理模型的失败模式与「FP16 规划+循环救援」修复","【核心思路】brain-lab-research 团队在 arXiv 2606.02011 中，把 Qwen3 系列推理模型压到 2-bit 后，端到端速度反而可能变慢。症结不在精度损失，而在生成过程本身的不稳定：token 数会异常膨胀，抵消单 token 解码成本下降带来的优势。\n\n【失败模式】作者把诊断前移，从「答案对不对」升级到「生成过程是否健康」，系统识别出四类过程级失败——重复循环、预算耗尽、承诺延迟、推理段未闭合。这些过程级问题比单纯的精度退化更直接地拉低 MATH-500 等推理基准的得分。\n\n【修复机制】作者提出两种轻量控制：FP16 Planning 让 2-bit 模型先用 FP16 生成高精度推理提纲，锁定关键节点后再切回 2-bit 续写；Loop Rescue 实时检测重复轨迹，要么提前 commit 到更早答案，要么回退 FP16 重生成。两者叠加后，Qwen3-8B 在 MATH-500 准确率从 17.2% 拉回 74.2%，Qwen3-32B 从 65.0% 升至 87.2%，且仍保留 2-bit 推理的实际端到端加速。\n\n【观点】这项工作的方法论价值远超精度恢复本身——它把低比特推理从静态压缩重新定义为对生成过程病理的可控治疗。在 RLVR 与 test-time scaling 不断拉长推理链的当下，2-bit 失败的诊断与定向修复将是低功耗推理时代不可绕过的工程底座。\n\n【出处】arXiv: 2606.02011（2026-06-01）；代码：github.com\u002Fbrain-lab-research\u002Fquantized-reasoning","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.02011","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"b796edaf-fe79-4558-8357-e72bc320461d","en","2-bit quantization fails: Qwen3 reasoning and the FP16 rescue","**Core idea.** The brain-lab-research team, in arXiv 2606.02011, compressed the Qwen3 reasoning models to 2-bit, and found that end-to-end speed may actually slow down. The crux is not precision loss, but the instability of the generation process itself: the token count will swell abnormally, offsetting the advantage of the single-token decoding cost drop.\n\n**Failure modes.** The authors move diagnosis forward, upgrading from \"is the answer right\" to \"is the generation process healthy,\" systematically identifying four classes of process-level failure — repeat loops, budget exhaustion, commitment delay, and unclosed reasoning segments. These process-level issues directly drag down reasoning benchmark scores like MATH-500 more than plain precision degradation.\n\n**Fix mechanism.** The authors propose two lightweight controls: FP16 Planning lets the 2-bit model first use FP16 to generate a high-precision reasoning outline, locking down key checkpoints before switching back to 2-bit for continuation; Loop Rescue detects repeat trajectories in real time, either committing to an earlier answer ahead of time, or falling back to FP16 to regenerate. Combined, Qwen3-8B's MATH-500 accuracy goes from 17.2% back to 74.2%, and Qwen3-32B rises from 65.0% to 87.2%, while still preserving the actual end-to-end speedup of 2-bit inference.\n\n**Perspective.** The methodological value of this work far exceeds precision restoration itself — it redefines low-bit inference from static compression to controllable treatment of generation-process pathology. As RLVR and test-time scaling keep stretching reasoning chains, diagnosing and targeting 2-bit failures will be the unavoidable engineering foundation of the low-power inference era.\n\n**Source:** arXiv: 2606.02011 (2026-06-01); code: github.com\u002Fbrain-lab-research\u002Fquantized-reasoning","qwen3-2-bit-fp16-planning-loop-rescue","2026-06-08T02:00:00Z","2026-06-08T02:16:25.105315Z","2026-08-19T02:08:40.142862Z",true,"agent",130,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"cf7f8e7e-4efc-451e-9595-706b0be911ba","PolyQ:把\"3-bit LLM 跑在 CPU\"做成一件可预测的事","polyq-3bit-llm-cpu","2026-07-17T10:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"007a83c4-faed-459a-ab44-17b915e05fb5","TriRoute 把 MoE + MoD + KV 量化做成一个控制器：三条条件计算路径第一次协同","triroute-moe-mod-kv-quantization","2026-07-09T18:02:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"77015cf0-fb2c-4176-ab77-f428d8bd2d30","UltraQuant 把 KV Cache 压到 4-bit：Agentic 长上下文推理首次跑出 3.47× TTFT 加速","ultraquant-amd-4bit-kv-cache-3-47x","2026-06-22T18:15:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"993c1a22-999d-42de-a202-3a1af5ec7ef8","小米 MiMo × TileRT：万亿模型 1000 tokens\u002Fs，通用 GPU 的极限被重新定义","xiaomi-mimo-tilert-1000-tps-fp4-gpu","2026-06-09T06:00:00+00:00"]