[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-kashin-dct-2bit-llm-quantization":3,"topics-all":38,"news-related-e91b3add-4f1d-48d3-ab7a-bd2c6e8c1765":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"e91b3add-4f1d-48d3-ab7a-bd2c6e8c1765","QuIP 崩、OPTQ 降级:Kashin-DCT 在 4-bit 量化压力测试里活了下来","俄罗斯团队改进 Kashin 分解量化:符号随机化 DCT 替换稠密正交矩阵,迭代成本 O(N²) 降为 O(N log N),聚类中心闭式初始化。4-bit 下与 OPTQ、QuIP 系打平,在 QuIP 发散或 NaN 中止的压力配置中保持数值稳定;权重分解为两个 2-bit 因子码,适配原生 2-bit 硬件。","LLM 推理的成本大头是权重显存,后训练量化(PTQ)不重训模型就能把权重压到 4-bit 甚至更低,是绝大多数部署团队的现实选择。但主流方法有个不太被讨论的软肋:数值稳定性——不是平均分不够高,而是某些层会突然炸掉。9 月 10 日提交到 arXiv 的一篇论文([arXiv:2609.11687](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.11687))把一条 1977 年的数学老路线认真工程化了一遍,作者来自 Institute of Numerical Mathematics 与 Steklov 数学研究所,名单里就有定理提出者 Boris Kashin 本人(1977 年的分解定理即以他命名)。\n\n## 先看翻车现场\n\n论文的对比实验里,QuIP 家族的不稳定不是偶发:在 Pythia-6.9B 上,QuIP 与 QuIP-RG 的 WikiText-2 困惑度直接发散到 2000 以上,关掉微调与向量码本的 QuIP# 变体也飙到约 325,而 Kashin-DCT+H 收在 20.6±1.6,比同预算的 QuIP# 好一个数量级以上;同一配置下 OPTQ 尚能贴近自身基线,约 12.02,说明炸的是方法不是模型。Mistral-7B v0.1 上更尖锐:四个 QuIP 变体全部在 SwiGLU 的 mlp.down_proj 层因 LDL 回代出现 NaN 而中止,OPTQ 也退化到约 380 困惑度;Kashin-DCT 保持数值稳定,距 FP16 基线 8.63 只差约 0.3。按论文报告,在他们遇到的全部四个压力配置里,这是唯一每层都保持稳定的 pipeline。\n\n## 三处手术\n\n方法本身是对 Kashin 分解量化的三项改造:\n\n- **稠密正交矩阵换成符号随机化的 DCT**。原方案要存一个 N×N 随机正交矩阵,单次迭代 O(N²);换成带随机符号的离散余弦变换后成本降到 O(N log N),矩阵本身零存储。\n- **交替更新的分块贪心算法**。每四步固定两个因子的更新顺序,保证它们都呈现稳定 2-bit 聚类所需的四峰分布,收敛有几何衰减证明,不再是\"经验上碰运气\"。\n- **聚类中心闭式初始化**。峰值位置 ±c1±c2 可由残差范数解析算出,直接用来初始化 k-means,砍掉了此前主导量化耗时的多次重启搜索。\n\n外围再叠上 OPTQ 式顺序误差补偿与 QuIP 式 incoherence 预处理(Hadamard 或 Kronecker 两种),整条流水线用 JAX 实现,支持多 GPU。\n\n## 结果:稳,且不牺牲平均分\n\n4-bit per channel 配置下,在 OPT-1.3B、Llama-2-7B\u002F13B、Pythia-1.4B 上,该方法与 OPTQ、QuIP、QuIP-RG 以及去微调版 QuIP# 基本打平:Llama-2-7B 的 WikiText-2 困惑度,FP16 基线 9.20,Kashin-DCT+H 为 9.60,略优于 OPTQ 的 9.80;QuIP 在同一配置上方差大到 48.30±29.84。规模往上走同样成立:Llama-2-13B 上 FP16 基线 8.11,去微调版 QuIP# 为 8.31,Kashin-DCT+H 为 8.37,差距都在零点几个点以内。推理时每个权重分解为两个 2-bit 因子码,结构上适配原生 2-bit 硬件。实验在单张 H100 上完成,校准用 1000 条长度 2048 的 WikiText-2 序列,评测走 lm-evaluation-harness,表中数字为三个随机种子下的均值与方差。\n\n论文也明说局限:没有与 AWQ、OmniQuant、旋转类激活量化方法对比,只论证了旋转预处理与本方法的分解是正交可组合的。\n\n## 所以呢\n\n这篇论文值得注意的点不在刷分——它也没刷分——而在把\"最坏情况不炸\"摆到了和\"平均困惑度\"同等的位置。部署侧真正致命的从来不是 0.2 的困惑度差距,而是某一层静默 NaN 之后整个服务不可用。当 2-bit 原生硬件真的到来,先活下来的方法,才轮得到谈精度。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.11687","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e0e2f97f-5ceb-40f1-a32e-2b06d48ee3b7","en","Kashin-DCT quantization stays stable where QuIP hits NaN at 4-bit","A Russian numerical-mathematics team revisits Kashin-decomposition quantization for LLMs: a sign-randomized DCT replaces the dense random orthogonal matrix, cutting per-iteration cost from O(N^2) to O(N log N), and closed-form k-means initialization removes multi-restart clustering. At 4-bit it matches OPTQ and QuIP-family baselines while staying numerically stable in stress configs where QuIP diverges or aborts with NaN; each weight decomposes into two 2-bit factor codes suited to native-2-bit hardware.","The dominant cost of LLM inference is weight memory. Post-training quantization (PTQ) compresses weights to 4-bit and below without retraining, which makes it the pragmatic choice for most deployment teams. But mainstream methods share an under-discussed weakness: numerical stability -- the problem is not the average score, it is that certain layers suddenly blow up. A paper submitted to arXiv on September 10 ([arXiv:2609.11687](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.11687)) seriously engineers a mathematical route dating back to 1977. The authors come from the Institute of Numerical Mathematics and the Steklov Mathematical Institute, and the author list includes Boris Kashin himself -- the 1977 decomposition theorem carries his name.\n\n## The crash sites\n\nIn the paper's comparison experiments, QuIP-family instability is not incidental. On Pythia-6.9B, QuIP and QuIP-RG diverge beyond 2000 WikiText-2 perplexity, the QuIP# variant with fine-tuning and the vector codebook disabled still blows up to roughly 325, while Kashin-DCT+H lands at 20.6±1.6 -- more than an order of magnitude better than QuIP# at the same bit budget. Mistral-7B v0.1 is sharper still: all four QuIP variants abort with NaN in LDL back-substitution on the SwiGLU mlp.down_proj layer, and OPTQ itself degrades to roughly 380 perplexity; Kashin-DCT stays numerically stable, within about 0.3 of the FP16 baseline of 8.63. Per the paper's own reporting, across all four stress configurations they encountered, this is the only pipeline that stays stable on every layer.\n\n## Three surgical changes\n\nThe method is a rebuild of Kashin-decomposition quantization with three fixes:\n\n- **Dense orthogonal matrix replaced by a sign-randomized DCT.** The original scheme stores an N×N random orthogonal matrix with O(N²) per-iteration cost; a sign-randomized Discrete Cosine Transform cuts this to O(N log N) with zero stored matrix.\n- **A partitioned greedy algorithm with alternating updates.** Fixing the update order in blocks of four guarantees the four-peak distributions required for stable 2-bit clustering of both factors, with a proven geometric convergence rate -- no longer luck of the draw.\n- **Closed-form cluster-center initialization.** Peak locations ±c1±c2 are known analytically from residual norms and directly initialize k-means, eliminating the multi-restart search that previously dominated quantization wall-clock time.\n\nAround this core, the pipeline composes OPTQ-style sequential error compensation and QuIP-style incoherence preprocessing (Hadamard or Kronecker), implemented in compile-friendly JAX with multi-GPU support.\n\n## Results: stable, without giving up the average\n\nAt 4-bit per channel on OPT-1.3B, Llama-2-7B\u002F13B, and Pythia-1.4B, the method is competitive with OPTQ, QuIP, QuIP-RG, and a fine-tuning-free variant of QuIP#. On Llama-2-7B WikiText-2 perplexity: FP16 baseline 9.20, Kashin-DCT+H at 9.60, slightly better than OPTQ's 9.80; on the same configuration QuIP's variance reaches 48.30±29.84. At inference time each weight decomposes into two 2-bit factor codes per channel, structurally suited to native-2-bit hardware. Experiments ran on a single H100, calibrated on 1000 sequences of length 2048 from WikiText-2, evaluated through lm-evaluation-harness.\n\nThe paper is upfront about limitations: no comparison against AWQ, OmniQuant, or rotation-based activation-quantization baselines; it only argues that rotation preprocessing composes orthogonally with this factorization.\n\n## So what\n\nThe notable part of this paper is not benchmark chasing -- it does none -- but elevating \"worst case does not explode\" to the same rank as \"average perplexity\". What kills deployments is never a 0.2 perplexity gap; it is one layer silently returning NaN and the whole service going down. When native 2-bit hardware actually arrives, the methods that survive first are the ones that get to talk about accuracy at all.","kashin-dct-2bit-llm-quantization","2026-09-12T15:10:00Z","2026-09-12T15:11:55.227397Z","2026-09-12T15:11:55.227405Z",true,"agent",111,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"5a5b1531-e1b2-469b-8064-772223231183","KronQ：Kronecker Hessian 拆掉 GPTQ 的 2-bit 墙","kronq-kronecker-hessian-gptq","2026-07-13T16:02:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"1480f5c1-5eea-4513-bb38-ad5a4bb3cc25","Log_bQuant 改写 4-bit 量化:TUM 让 14B LLM 保住 72.97% MMLU","log-bquant-4bit-quantization","2026-07-06T20:11:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"5e7da1f9-83c8-421d-bd6b-0a2957bfba76","Mamba-2 也撑不住 1.58-bit：从预训练 checkpoint 出发，QAT 把 SSM 压到 744MB","mamba-2-1-58-bit-qat-744mb-102m-tokens","2026-06-18T06:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"4978562a-00d5-4230-adbc-821bf89b08f5","EdgeRazor：1.58比特精度极限压缩，大模型边缘部署迎来新解法","edgerazor-1-58-bit-nanjing-microsoft-qwen","2026-05-07T22:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"2266cea6-06f1-4932-8905-1bc3f2e5a8c0","Meta FAIR 字节蒸馏研究:End-Of-Token 渐近反超 token 蒸馏 4%,数据只需 1\u002F6","meta-fair-byte-distillation-token-ceiling-2026-09","2026-09-15T02:00:00+00:00"]