[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-codequant2-reasoning-model-quantization":3,"topics-all":38,"news-related-8c0d584e-4bb2-4d00-a5cd-9529824f756d":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"8c0d584e-4bb2-4d00-a5cd-9529824f756d","量化推理模型先管开头30步:CodeQuant2","CodeQuant2 专治推理模型低比特量化崩盘:它发现推理前 30 个解码步的激活波动比非思考解码高约 20%,据此用推理感知校准+自适应码本聚类,在 Qwen3-4B-Thinking 上把 A4W4 平均分从 0.664 提到 0.770、AIME25 pass@1 拉回 0.563,平均提速 2.42 倍。","把 Qwen3-4B-Thinking 压到 4-bit 有多惨?BF16 下它的 AIME25 pass@1 是 0.688,换旋转系经典方法 QuaRot 在 A4W4 设置下跑,直接掉到 0.004——近乎归零。推理模型靠长思维链吃饭:量化的近似误差会沿着自回归解码一步步累积,开头错一点,后面整条推理链跟着崩。所以低比特量化推理模型,一直比量化普通模型难啃;而硬件又在推着大家往低比特走,Hopper\u002FAda 支持 FP8,Blackwell 进一步支持 FP4,矛盾只会更尖锐。\n\n## 病根:推理的前 30 步最不稳定\n\narXiv 2611.09868 的作者做了一个朴素但关键的观察:在 GSM8K、AIME25 这类推理任务上,模型生成思维链时,前 30 个解码步的平均激活标准差比非思考解码高约 20%;通道级离群值在第 1 步从浅层贯穿到深层,到第 400 步已明显减弱。而常规校准流程喂的是普通文本,根本覆盖不到这种“开局剧烈波动”的分布——校准集与部署分布错位,是推理模型量化崩盘的第一个病根。\n\n## 药方:四个组件加一个查表 kernel\n\nCodeQuant2 是一套聚类式 LUT 量化框架,四个组件各管一段:\n\n- **旋转+缩放抑制离群值**:共享正交旋转矩阵配层级对角缩放,经 Cayley 变换保持正交性,只作用于 QKV、gate、up 投影;\n- **自适应簇分配(ACA)**:候选簇数 K∈{8,16,32},用一阶近似的重要性分数判断哪个矩阵该加容量,再拿实测 LUT 延迟当预算约束——敏感矩阵给 32 簇,不敏感的降到 8 簇,平均指派索引宽度保持在 4-bit;\n- **可学习聚类**:K-means 初始化后冻结硬指派,用低秩(LoRA 式)更新学软指派,温度退火逐渐变硬;\n- **推理感知校准(RAC)**:让全精度模型在 GSM8K 训练集上先生成 256 个 token,再与 WikiText2 按 9:1 混合,校准序列截断或补齐到 512 token,让校准集里真有“推理开头的激活”。\n\n推理侧则是把解码期矩阵乘法改写成查表:4-bit 激活层级与权重质心的乘积预计算成查找表,常驻 SM 的 shared memory;激活的 min\u002Fmax 计算、量化、位打包融合成一次读取,并针对解码期 batch 并行有限的问题调整 tile 尺寸。\n\n## 疗效:崩盘项拉回来,速度还快了\n\nQwen3-4B-Thinking 在 A4W4 下,六项任务平均分从 CodeQuant 的 0.664 提到 0.770(BF16 为 0.838),论文口径是 10.6 个百分点;最崩的 AIME25 pass@1 从 0.392 拉回 0.563,pass@8 从 0.500 到 0.800;LiveCodeBench 比 CodeQuant 高 15 个百分点。消融很干净:四个组件分别贡献 +4.00、+3.64、+1.94、+1.08,没有一个凑数。系统侧,在 A6000、L40s、A100 三块卡上对比 BF16 平均提速 2.42 倍,最高 2.85 倍,A6000 上 batch 16 仍有 2.28 倍。整套流程单张 H100 就能跑,三段微调各需约 30、15、30 分钟。\n\n当然收益不均匀:Qwen3-8B 和 Phi-4-Reasoning 上的提升温和得多,8B 的 AIME25 pass@1 只从 0.446 到 0.479。模型越大、任务越不极端,量化修复的边际收益越小,这个规律没变。\n\n## 所以呢\n\n这篇论文最值得带走的不是某个数字,而是一个方法论转向:**量化校准集应该长得像你的部署负载**。跑推理负载,就用推理开头的 token 做校准;跑普通对话,普通文本就够。另一个启示是 ACA 的思路——码本容量是精度-延迟的预算问题,不是精度最大化问题:8 簇换 47 token\u002Fs,32 簇只剩 33,自适应分配拿到 44,几乎吃到 16 簇档的速度还换来更高的分。给每个矩阵发对工资,比全员涨薪划算。论文:https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.09868","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.09868","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"2ff790d9-f30a-4d13-b040-b36ce66a05ee","en","CodeQuant2: Fixing 4-bit Collapse in Reasoning LLMs","CodeQuant2 fixes 4-bit collapse in reasoning LLMs via reasoning-aware calibration + adaptive clustering: Qwen3-4B A4W4 avg 0.664→0.770, 2.42x.","How badly does Qwen3-4B-Thinking fare when squeezed to 4-bit? At BF16 it scores 0.688 pass@1 on AIME25; under the A4W4 setting with the classic rotation-based method QuaRot, the score collapses to 0.004 — effectively zero. Large reasoning models live on long chains of thought: quantization approximation errors accumulate step by step through autoregressive decoding, and a small error early on can wreck the entire reasoning chain. Low-bit quantization for reasoning models has always been harder than for ordinary models — and hardware keeps pushing everyone toward lower precision: Hopper and Ada GPUs support FP8 arithmetic, while Blackwell extends it to FP4, so the tension will only sharpen.\n\n## The diagnosis: the first 30 decoding steps are the wildest\n\nThe authors of arXiv 2611.09868 made a deceptively simple observation: on reasoning benchmarks such as GSM8K and AIME25, when the model generates a thinking trace, the average activation standard deviation over the first 30 decoding steps is about 20% higher than that of non-thinking decoding. Channel-wise outliers at step 1 run from shallow layers all the way through to deeper ones, and by step 400 the outlier pattern is noticeably weaker. Conventional calibration pipelines feed on generic text and never cover this \"violent opening\" distribution — a calibration set that diverges from the deployment distribution is the first root cause of reasoning-model quantization collapse.\n\n## The fix: four components plus a lookup kernel\n\nCodeQuant2 is a clustering-based LUT quantization framework in which four components each own one piece of the problem:\n\n- **Rotation + scaling for outlier suppression**: a shared orthogonal rotation matrix paired with layer-wise diagonal scaling, optimized through the Cayley transform to preserve orthogonality, applied only to the query, key, value, gate, and up projections;\n- **Adaptive Cluster Allocation (ACA)**: candidate cluster counts K∈{8,16,32}; a first-order importance score decides which matrix deserves extra capacity, while profiled LUT latency serves as the budget constraint — sensitive matrices get 32 clusters, insensitive ones drop to 8, keeping the average assignment-index width at 4 bits;\n- **Learnable clustering**: after K-means initialization the hard assignments are frozen, soft assignments are learned via LoRA-style low-rank updates, and a temperature schedule anneals them back to hard assignments;\n- **Reasoning-aware calibration (RAC)**: let the full-precision model first generate 256 tokens on the GSM8K training split, mix with WikiText2 at 9:1, and truncate or pad each calibration sequence to 512 tokens, so the calibration set genuinely contains \"the activations of a reasoning opening.\"\n\nOn the inference side, decode-time matrix multiplication is recast as table lookup: products between 4-bit activation levels and weight centroids are precomputed into a lookup table that lives in each SM's shared memory; activation min\u002Fmax computation, quantization, and bit packing are fused into a single read, and tile sizes are retuned for the limited batch-level parallelism of decode-time GEMM.\n\n## The results: collapsed items come back, and it is faster\n\nOn Qwen3-4B-Thinking under A4W4, the average score over six task benchmarks rises from CodeQuant's 0.664 to 0.770 (BF16 is 0.838) — a 10.6-point gain in the paper's accounting. The most collapsed item, AIME25 pass@1, climbs from 0.392 back to 0.563; pass@8 goes from 0.500 to 0.800; LiveCodeBench beats CodeQuant by 15 points. The ablation is clean: the four components contribute +4.00, +3.64, +1.94, and +1.08 respectively — none of them is filler. On the systems side, across A6000, L40s, and A100 GPUs, CodeQuant2 reaches an average 2.42x speedup over BF16, peaking at 2.85x, and still retains at least a 2.28x speedup at batch size 16 on A6000. The whole pipeline fits on a single NVIDIA H100, with the three fine-tuning stages taking roughly 30, 15, and 30 minutes. Evaluation was run with EvalScope, four-shot for GSM8K and zero-shot elsewhere, with per-channel weight and per-token activation granularity kept identical across all compared methods.\n\nThe gains are not uniform, though: on Qwen3-8B and Phi-4-Reasoning the improvements are more modest — the 8B model's AIME25 pass@1 moves only from 0.446 to 0.479. The larger the model and the less extreme the task, the smaller the marginal return of quantization repair; that regularity has not changed.\n\n## So what\n\nThe most valuable takeaway from this paper is not any single number but a methodological shift: **your quantization calibration set should look like your deployment workload**. Running reasoning workloads? Calibrate on tokens from reasoning openings. Running ordinary chat? Generic text is enough. The other lesson is ACA's framing — codebook capacity is an accuracy-latency budget problem, not an accuracy-maximization problem: 8 clusters buys 47 token\u002Fs, 32 clusters leaves only 33, and adaptive allocation lands at 44 token\u002Fs, nearly the speed of the 16-cluster tier with a higher score to show for it. Paying each matrix the right salary beats giving everyone a raise. Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.09868","codequant2-reasoning-model-quantization","2026-11-09T17:10:11Z","2026-11-09T17:10:12.141441Z","2026-11-09T17:10:12.141449Z",true,"agent",289,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"b80697dc-e9ee-40f5-a2c7-3944cd36abba","权重不等长:哈夫曼重编码,边缘LLM再省31.4%","lazyweights-huffman-edge-llm","2026-11-13T23:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"081148f2-39eb-432d-8841-5cb1e58ed527","省内存还是多跑几遍:本地小模型怎么花推理预算才划算","quantization-bits-vs-option-order-budget","2026-11-12T09:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"a6f5abcc-f1eb-4b6c-ba50-062836f50d0d","量化前先算一遍:二阶筛查排序能信,绝对值不能","quantization-fisher-screen-pre-registered-audit","2026-11-11T03:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"62bc171e-2954-4c08-b3e8-d484acce7894","回放AUC 0.93,实跑却分不出:本地小模型升位门控","uncertainty-gate-local-llm-replay-vs-live","2026-11-10T17:57:51+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"93963f65-e9a2-4058-ba37-563154d1aebd","bf16 状态:推理模型准确率 85.6 掉到 70.8","bf16-state-cache-hybrid-reasoning","2026-11-10T07:15:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"8b9f54f1-334a-4725-9c50-43dd1f018fba","TWIG:把权重存成一棵树,量化推理全打通","twig-weight-tree-llm-quantization","2026-10-27T01:30:00+00:00"]