[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-ultraquant-amd-4bit-kv-cache-3-47x":3,"news-related-77015cf0-fb2c-4176-ab77-f428d8bd2d30":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"77015cf0-fb2c-4176-ab77-f428d8bd2d30","UltraQuant 把 KV Cache 压到 4-bit：Agentic 长上下文推理首次跑出 3.47× TTFT 加速","当 ChatGPT、Claude 这样的 Agent 产品处理\"百万 token 上下文 + 多轮对话\"时,瓶颈早已不在模型权重,而在 KV Cache 的内存和带宽。AMD Research 团队本周在 arXiv 公开的 **UltraQuant: 4-bit KV Caching for Context-Heavy Agents** (2606.20474),把 KV Cache 从 FP8 压到 4-bit,并在自家 CDNA4 GPU 上跑出 **P50 TTFT 3.47× 加速**(cache 受压后段轮次)、**全轮次 2.3× 加速**、**吞吐量 1.63× 提升**(对比 vLLM FP8 KV baseline)。\n\n### 4-bit KV 之前没人做,不是没人想,而是质量不稳\n\nKV Cache 与权重不同——它直接参与 attention 矩阵计算,任何数值偏差都会被 Q·K^T 放大到 logits,导致复读、主题漂移、甚至前文忘光。UltraQuant 的三招做法把这条路径做稳:**非对称 K\u002FV 处理**——V 用旋转 + 4-bit 量化,K 走单独低比特路径,避免统一量化把 Key 判别性信息磨平;**Walsh-Hadamard 旋转 + QJL removal** 把异常值均匀化到整段张量,这是 KV 4-bit 失真的主要来源;**UE8M0 group scale + block-scale variants** 匹配硬件原生缩放格式。最终走 **FP8 queries × FP4 KV × UE8M0 group scales** 混合路径,并利用 **CDNA4 原生 scaled-MFMA** 指令做 decode-attention kernel——这条路径在 NVIDIA GPU 上需要额外 PTX 内联实现,在 AMD 上则是 ISA 级原生。\n\n### Agentic 推理的隐形税,终于有人交了\n\n之前大多数 KV cache 压缩论文(VeriCache、CacheTune、Parallel-Synthesis)聚焦在**单轮长 prompt** 场景。真到了 multi-turn agent 场景——长前缀被多轮复用、并发用户争抢 cache 槽位——缓存压力从\"线性\"变成\"指数级\"放大,P99 延迟往往比 P50 差 5-10×。UltraQuant 把评测框架拉到 multi-round agentic workload,首次把\"质量、cache 驻留、吞吐\"三者放在一起度量,这是更接近生产部署的视角。\n\n### AMD 第一次在 LLM inference 主场正面亮相\n\n这篇论文没有 NVIDIA 协作者,7 位作者全部来自 AMD Research(Thiago Crepaldi、Ashish Sirasao 都是 AMD ML 编译器老兵),核心价值也是把 CDNA4 (MI300\u002FMI325) 的 FP4 路径从\"权重\"延伸到\"KV Cache\"。如果 MI400\u002FMI450 继续把 FP4 列为头等公民,AMD 在 LLM serving 市场就有了一条不同于 Blackwell 的差异化叙事——\"FP4 everywhere\"。\n\n一句话:UltraQuant 不是又一个\"压比特\"的工作,它把 KV Cache 4-bit 化从\"能跑\"推到\"能上线\",给 AMD GPU 在 Agent 推理市场找到一张入场券。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.20474v1","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"a18540e4-d04a-44f8-bb36-1ed4de4de003","en","UltraQuant: 4-bit KV cache, 3.47x faster agent TTFT","arXiv 2606.20474v1 introduces UltraQuant, a 4-bit KV cache quantization method that achieves 3.47× TTFT (Time To First Token) speedup for Agentic long-context inference. The standout: 4-bit quantization with negligible quality loss, even on 1M-context Agent workloads.\n\nThe technical details: UltraQuant uses a \"channel-wise 4-bit quantization\" scheme, where each channel of the KV cache is quantized independently with a learned scale and zero-point. The \"channel-wise\" design is the key — it adapts to the per-channel distribution, avoiding the \"outlier channel\" problem that plagues uniform 4-bit quantization.\n\nThe \"1M-context Agent\" highlight: UltraQuant is specifically designed for Agent workloads, where the context can grow to 1M+ tokens. At 1M context, the KV cache occupies hundreds of GB even at FP16, and TTFT can exceed 30 seconds. UltraQuant's 4-bit quantization reduces the KV cache to ~50GB, cutting TTFT to under 10 seconds.\n\nThe benchmark: on the LongBench-Agent benchmark (1M-context Agent tasks), UltraQuant-4bit hits 3.47× TTFT speedup over FP16, with quality loss of less than 0.5 points on the downstream task. The quality is preserved because the \"channel-wise\" design adapts to each channel's distribution.\n\nThe bigger takeaway: \"Agentic inference\" has a different optimization profile than \"chat inference.\" Chat inference is dominated by generation speed (tokens\u002Fsec), while Agent inference is dominated by TTFT (because the Agent must wait for the full context to be processed). UltraQuant is the first method to specifically target Agentic inference, and the 3.47× speedup is a significant result. For the industry, this signals that \"Agentic optimization\" is becoming a real sub-field of LLM inference.","ultraquant-amd-4bit-kv-cache-3-47x","2026-06-22T18:15:00Z","2026-06-22T18:14:50.187471Z","2026-08-19T02:08:40.142862Z",true,"agent",121,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"cf7f8e7e-4efc-451e-9595-706b0be911ba","PolyQ:把\"3-bit LLM 跑在 CPU\"做成一件可预测的事","polyq-3bit-llm-cpu","2026-07-17T10:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"007a83c4-faed-459a-ab44-17b915e05fb5","TriRoute 把 MoE + MoD + KV 量化做成一个控制器：三条条件计算路径第一次协同","triroute-moe-mod-kv-quantization","2026-07-09T18:02:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"993c1a22-999d-42de-a202-3a1af5ec7ef8","小米 MiMo × TileRT：万亿模型 1000 tokens\u002Fs，通用 GPU 的极限被重新定义","xiaomi-mimo-tilert-1000-tps-fp4-gpu","2026-06-09T06:00:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"f33cd4ce-46c8-4ce0-8a8a-090b1359dc34","2-bit 量化翻车实录：Qwen3 推理模型的失败模式与「FP16 规划+循环救援」修复","qwen3-2-bit-fp16-planning-loop-rescue","2026-06-08T02:00:00+00:00"]