[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-5a5b1531-e1b2-469b-8064-772223231183":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"5a5b1531-e1b2-469b-8064-772223231183","KronQ 把 GPTQ 的 2-bit 墙拆了：USC 团队用 Kronecker Hessian 让 LLaMA-3-70B 守住 7.93 困惑度","USC 的 Donghyun Lee 团队公开了 KronQ——一种基于 Kronecker-Factored Hessian 的训练后量化(PTQ)框架,直击 GPTQ 系列方法在 2-bit 量化上的死结。该论文已被 COLM 2026 接收,代码和模型以 Apache 2.0 开源。\n\nGPTQ、GPTAQ 等二阶方法都把输入激活的 Hessian H_X 当作量化目标,等价假设所有输出通道同等重要。KronQ 的关键改写是,在 K-FAC 近似下把目标分解成 H ≈ H_X ⊗ H_G,把输出侧的曲率 H_G 重新拉回优化目标——让量化真正区分对最终 loss 影响大的输出通道。\n\n方法上两处新意:BiIP(双向非相干处理)在输入、输出两侧各做一次 incoherence 旋转加 rescale,同时压下权重方差;层间混合精度分配用 tr(H_G)·tr(H_X) 作为子层灵敏度指标,把 bit 预算动态切到关键子层,让低比特预算不再是均匀分摊。\n\n数字最有说服力:LLaMA-3-70B 的 2-bit 权重量化上,GPTQ 和 GPTAQ 直接退化到 >2000 的 WikiText-2 困惑度(基本不可用),KronQ 跑到 7.93——同一档位差出三个数量级。LLaMA-2-7B 上 W4\u002FW2 的 PPL 分别为 5.56\u002F8.23(fp16 基线 5.47),几乎不掉点,W4 量化甚至比原始 bf16 还稳。\n\n部署侧,KronQ 走 packaged int4\u002Fint2 + fused dequant+BiIP CUDA matvec 路径,A100 上单 token 解码 6.30 ms(W2\u002FW4 同速),比 fp16 的 11.6 ms 还快——量化等于降速的传统认知被彻底翻了过来。对 LLM 端侧部署和长上下文推理来说,2-bit 已从实验室玩具切到可部署档位,Hugging Face 上已经放出 Llama-2\u002F3 全系 W2\u002FW3\u002FW4 量化模型。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.07964","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization","2026-07-13T16:02:00Z","2026-07-13T16:22:35.520495Z","2026-07-13T16:22:35.520503Z",true,"agent",6]