[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-reskv-residual-kv-cache-compression":3,"news-related-d2c56430-f9a7-4844-bfab-a8651279a70c":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"d2c56430-f9a7-4844-bfab-a8651279a70c","ResKV 不再把 KV 缓存压缩等同于删词：给被淘汰的信息留一份残差账本","ResKV 将固定 KV 槽位拆成精确主缓存与紧凑残差缓存，让被淘汰 token 的聚合贡献重新进入同一次 softmax。论文在 LLaMA-3.1-8B 与 Qwen-2.5-7B 上报告：LongBench 展示的 32 组配置全部提升，RULER 64 组中提升 63 组；优势尤其集中在紧预算和未知未来查询的场景。","# ResKV 不再把 KV 缓存压缩等同于“删词”：给被淘汰的信息留一份残差账本\n\n长上下文推理有个绕不开的成本：模型每生成一个新 token，都要回看历史 token 的 Key 和 Value。上下文越长，KV cache 占用的显存和内存带宽就越高。多数压缩方法因此先给历史 token 打分，只保留一小部分，其余直接淘汰；另一些方法把被淘汰的状态合并进保留项。前者省得干脆，却把被删除 token 对注意力的贡献一起抹掉；后者保住部分信息，却可能改动本应精确保留的 Key 和 Value。\n\n7 月 31 日提交到 arXiv 的论文 **ResKV**，换了一个角度：被淘汰的信息不必逐个保存，但它们对 softmax 注意力分子和分母的总体贡献，可以被压缩成“残差统计量”。于是，在固定的 KV 槽位预算内，ResKV 把缓存拆成两部分：\n\n- **主缓存**保存优先级高的 token，Key 和 Value 保持原样；\n- **残差缓存**用少量条目概括被淘汰 token 的聚合贡献。\n\n关键不是在注意力算完后补一个修正项。主缓存和残差条目进入**同一次 softmax 归一化**，残差由此同时补回分子中的 Value 加权贡献和分母中的概率质量。它既没有把遗漏信息塞进保留 token，也没有增加总 KV 槽位数。[论文原文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.29591)\n\n## 两道控制，避免残差喧宾夺主\n\n残差信息并非每层、每个 KV head 都同样有用。ResKV 在缓存构建阶段，用一组拟合查询和验证查询尝试多个残差预算比例；只有当残差缓存能降低相对于完整缓存的注意力输出重建误差时，才给该层和 KV head 分配残差槽位，否则全部预算仍交给主缓存。\n\n解码阶段还有一道动态门控。论文用主缓存中最大的注意力权重衡量分布是否尖锐：如果某个查询已经高度集中在主缓存的少数条目上，就压低残差权重，避免模糊明确的检索峰值；如果注意力较分散，就允许残差承担更多概率质量，补回散落在上下文中的证据。\n\n## 固定预算下，提升集中在紧预算与未知查询\n\n实验使用 LLaMA-3.1-8B-Instruct 和 Qwen-2.5-7B-Instruct，在 LongBench 与 RULER 上测试，并覆盖保留 10%、20%、30%、40% KV 的预算。ResKV 被接到 AdaKV 和 SnapKV 上，与原基线使用相同的总槽位数。\n\n论文报告，LongBench 展示的 32 组配置全部提升，两个骨干模型的平均增益为 1.02 分；在只保留 10% 和 20% KV 时，平均增益分别为 1.43 和 1.17 分。RULER 展示的 64 组配置中有 63 组提升，平均增益为 3.38 分。更值得注意的是，**不知道未来查询内容就先压缩缓存**的 query-agnostic 设置，平均提升 4.54 分，高于 query-aware 设置的 2.22 分。这更接近真实服务：系统通常不能等下一条用户问题出现后，再重新决定旧上下文该留什么。\n\n组件消融也给出了清晰边界。在 10% KV、query-agnostic 设置下，去掉验证代理，会让两个代表性 RULER 任务分别下降 4.00 和 7.04 分；去掉动态门控，分别下降 4.80 和 4.48 分；把共享 softmax 改成主缓存与残差各自归一化，RepoBench-P 下降 3.00 分。\n\n## 这不是免费的加速，但方向更像“信息压缩”\n\n效率图显示，ResKV 的峰值显存与 SnapKV 基本重合，额外内存开销可忽略，并能在 128K 上下文维持稳定解码；代价是每一步都要计算残差分支和门控，因此吞吐低于 SnapKV。论文没有把它包装成零成本胜利：它用适度计算开销，换取固定显存预算下更完整的注意力信息。\n\nResKV 真正重要的判断是：**KV cache 压缩不该只有“留或删”这一个旋钮。**当长上下文里的证据分散时，单个 token 看似不重要，它们合在一起却可能决定答案。下一步值得关注的，不只是更聪明地挑 token，而是如何把被删掉的信息压成仍能参与计算的结构。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.29591","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"95989cc3-3aa2-43f8-82ea-73cd76bfec6a","en","ResKV keeps a residual ledger of evicted KV cache entries","ResKV divides a fixed KV-slot budget into an exact main cache and a compact residual cache, allowing omitted tokens’ aggregate contribution to re-enter the same softmax. On LLaMA-3.1-8B and Qwen-2.5-7B, the paper reports gains in all 32 displayed LongBench configurations and 63 of 64 RULER configurations, with larger benefits under tight budgets and query-agnostic compression.","# ResKV Stops Treating KV-Cache Compression as Token Deletion: A Residual Ledger for Omitted Context\n\nLong-context inference carries an unavoidable cost. For every newly generated token, a model attends to the Keys and Values of earlier tokens. As the context grows, the KV cache consumes increasing memory and memory bandwidth. Most compression methods therefore score historical tokens and retain only a subset, while others merge evicted states into retained entries. Hard eviction is efficient, but it also removes the evicted tokens’ aggregate contribution to attention. Merging preserves more information, yet it can perturb Keys and Values that should have remained exact.\n\n**ResKV**, submitted to arXiv on July 31, reframes the problem. Individual evicted tokens do not have to remain in the cache, but their aggregate contribution to the numerator and denominator of softmax attention can be represented as residual statistics. Under the same fixed number of KV slots, ResKV divides the cache into two parts:\n\n- an **exact main cache** that stores high-priority tokens without changing their Keys or Values;\n- a **compact residual cache** that summarizes the aggregate contribution of omitted tokens.\n\nThe residual is not applied as a correction after attention has already been computed. Main-cache tokens and residual entries participate in the **same softmax normalization**, allowing residual entries to restore both the value-weighted numerator and the probability mass in the denominator. The method neither folds omitted information into retained tokens nor increases the total KV-slot budget. [Read the paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.29591).\n\n## Two controls keep the residual from overwhelming exact retrieval\n\nResidual information is not equally useful in every layer or KV head. During cache construction, ResKV evaluates several candidate residual-budget ratios with fitting and validation queries. A layer and KV head receive residual slots only when the residual cache reduces attention-output reconstruction error relative to a full cache. Otherwise, the entire budget remains assigned to the exact main cache.\n\nA second control operates during decoding. ResKV measures attention sharpness using the largest attention weight over the main cache. If a query already forms a sharp retrieval peak over exact entries, the dynamic gate down-weights the residual logits to avoid diluting that peak. If main-cache attention is diffuse, residual entries retain more mass and can restore evidence distributed across the omitted context.\n\n## Gains are concentrated under tight budgets and unknown future queries\n\nThe experiments use LLaMA-3.1-8B-Instruct and Qwen-2.5-7B-Instruct on LongBench and RULER, covering retained KV budgets of 10%, 20%, 30%, and 40%. ResKV is attached to AdaKV and SnapKV while keeping the total number of retained KV slots identical to each baseline.\n\nThe paper reports improvements in all 32 displayed LongBench configurations, with an average gain of 1.02 points across the two backbones. At 10% and 20% retained KV, the average gains are 1.43 and 1.17 points. On RULER, ResKV improves 63 of the 64 displayed configurations, with an average gain of 3.38 points.\n\nThe more practical result appears in the **query-agnostic** setting, where the cache must be compressed before a future query is known. ResKV improves that setting by 4.54 points on average, compared with 2.22 points under query-aware construction. This resembles real serving more closely: a system usually cannot wait for the user’s next question before deciding which parts of the existing context to retain.\n\nThe ablations make the mechanism more concrete. Under a 10% retained-KV, query-agnostic setup, removing the validation proxy reduces two representative RULER task scores by 4.00 and 7.04 points. Removing the dynamic gate reduces them by 4.80 and 4.48 points. Replacing shared softmax with separate normalization for the main and residual caches lowers RepoBench-P by 3.00 points.\n\n## This is not free acceleration, but it is closer to information compression\n\nThe efficiency analysis shows that ResKV’s peak allocated memory essentially overlaps with SnapKV, indicating negligible extra memory overhead. It also maintains a stable decode rate through 128K context. The trade-off is explicit: evaluating the residual branch and dynamic gate at every decode step gives ResKV lower throughput than SnapKV. The paper does not present the method as a zero-cost improvement; it exchanges a moderate amount of computation for better preservation of attention information under a fixed memory budget.\n\nResKV’s central point is broader than one implementation: **KV-cache compression should not have only a “keep or delete” switch.** When evidence is distributed across a long context, individual tokens may look unimportant even though their aggregate contribution changes the answer. The next useful question is therefore not only which tokens to retain, but how omitted information can be compressed into a structure that still participates in inference.","reskv-residual-kv-cache-compression","2026-08-03T10:43:23Z","2026-08-03T10:57:37.697453Z","2026-08-03T10:57:37.697460Z",true,"agent",164,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"5f745fe5-ea5d-453a-8b08-7dac524d1ac2","ACL 2026 综述 sKis：KV 缓存优化重塑为 LLM serving 系统学","acl-2026-skis-kv-cache","2026-07-12T18:15:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"dd9d199c-5cd9-4a15-8ae6-7e4fb40f4129","MosaicKV:把 KV 缓存压成「马赛克」,长上下文推理跑出 16× 注意力加速","mosaickv-mosaic-compression","2026-07-03T18:01:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"c1ee21bd-4b59-418a-9b77-e46c790a8978","InfoKV 把 KV 缓存压缩推过「只看注意力」的临界点：用信息熵帮推理模型跑得更长","infokv-entropy-kv-cache-compression","2026-06-27T18:14:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"c4380194-5bf9-43a0-8460-46436a4f2f97","LCLM 把上下文压到 1\u002F16：8.8 倍提速的代价是 16 倍时准确率只剩 75%","lclm-1-16-compress-8-8x-75pct-accuracy","2026-06-15T04:00:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"94e7739c-f218-4dfc-803d-3662a97321f3","DeepSeek V4 混合注意力架构解析：如何在1M上下文下将计算量降至原来的27%？","deepseek-v4-csa-hca-1m-27pct-flops","2026-05-27T07:20:00+00:00"]