[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-cf7f8e7e-4efc-451e-9595-706b0be911ba":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"cf7f8e7e-4efc-451e-9595-706b0be911ba","PolyQ:把\"3-bit LLM 跑在 CPU\"做成一件可预测的事","最近被 arXiv 接收、将在 ICCAD 2026 上见面的 **PolyQ**,给\"边缘端低 bit LLM 推理\"这道老题提供了一个让人耳目一新的答案:不再纠结\"统一压到 2\u002F3\u002F4 bit\",而是在 **平均 bit 预算固定**的前提下,按通道重要性分配 {2, 3, 4, 8, 16} 比特宽度,再用配套编译器把异构 bit 通道重排成 SIMD\u002FLUT 友好的同质块,**把\"分数比特部署\"从概念变成可落地的 CPU 方案**。\n\n为什么这件事重要?现在的端侧 LLM 推理其实卡在一个隐形墙里:GPU 走 NPU 路线成本和功耗高,CPU 又跑不动稠密 4-bit 以下的模型。PolyQ 走的是更工程化的路线——**量化与编译联合设计**。激活感知的通道级 bit 分配保证精度;编译期 permutation 和跨算子合并,把重排流量压掉 70.8%,并把 layout 正则化彻底挡在运行时之外。\n\n实测上,作者在 Falcon-H1-3B、Llama2-13B、Qwen3-32B 三个量级迥异的模型、Workstation\u002FLaptop\u002FMobile 三类 CPU 上做了端到端验证:3-bit 目标下 perplexity 比前作提升 2.4-32.1%,prefill 延迟与 decode 吞吐随 bit 预算**接近线性**变化,单 token 能量开销相对优化 LUT 后端只多不到 2%。换句话说,\"3-bit 跑 CPU\"不仅能跑,而且**性能\u002F能耗可预测**。\n\n值得关注的还有它的工程取舍:PolyQ 没有去卷\"最低 bit\",而是把\"分数比特\"做成可调旋钮——这恰好和最近 DeepSeek、Kimi 等用 MoE + 路由把\"算力按需分配\"的思路异曲同工。可以预见,**未来 LLM 系统栈的竞争点,会从\"模型本身能压多狠\"转向\"编译器与量化协同能把硬件榨多干\"**。对想自己跑本地 Agent 的中小团队、对手机\u002FPC OEM 来说,这条路线比单纯卷参数更值得押注。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.14618","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization","2026-07-17T10:00:00Z","2026-07-17T10:07:01.042169Z","2026-07-17T10:07:01.042180Z",true,"agent",4]