[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-kv-cache-repair-budget-target-chunk":3,"topics-all":38,"news-related-53d00d00-e4fb-41b0-a36a-37161203b514":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"53d00d00-e4fb-41b0-a36a-37161203b514","KV缓存拼接复用丢题可至100%,修补要盯答案块","arXiv 新论文给非前缀 KV 缓存复用算细账:三个 7B-8B 模型 × 四类数据集上,复用丢掉 19%-100% 依赖上下文才能答对的题,干扰块越像答案块丢得越多。可修复的伤害散在干扰块里,小预算零敲碎打没用;把答案块对着复用缓存重算,已知靶块时 5% 预算在九个问答与 API 设置全胜 token 级修补器。","RAG、长期对话记忆、工具型智能体,每次请求都要把一批检索来的块拼进上下文。组合千变万化,前缀缓存基本派不上用场,于是工程上流行「非前缀 KV 复用」:每个块独立编码一次存下键值,查询时把缓存拼接起来,省掉对整个检索上下文的 prefill。省钱是真的,但代价此前没人细算——arXiv 上的新论文(2611.06397)把这笔账算清了:丢掉的题,最多能到全部。\n\n## 复用到底丢了多少\n\n作者在 Qwen3-8B、Llama-3.1-8B-Instruct、Mistral-7B-Instruct-v0.2 三个模型上,用 LoCoMo(对话记忆)、NQ-Open 与 TriviaQA(开放域问答)、Gorilla(API 文档)四类数据集,共 12 个设置,只统计「完整 prefill 能答对、且必须依赖检索上下文」的题。结果:复用丢掉其中 19%-100%。\n\n丢多少还取决于干扰块长什么样。周围放不相干的跨领域文本,损失仍在;换成同语料的无关块,12 个设置里 9 个显著变差;再换成与答案块相似的段落,六个问答设置全部进一步掉分,其中五个显著。此前工作里专门为「独立块编码」微调过的模型,在 NQ-Open 上仍丢 60%,且大部分损失同样来自相似块。论文里的直观例子:Qwen3-8B 完整 prefill 时答得对,复用同一批段落时,答案却被相似段落带偏,念出了相似段落里的女演员。\n\n## 修补预算该给谁\n\n现有修补方法分两派:token 级选择器(CacheBlend、EPIC、ProphetKV、CacheTune 等)挑少量散落 token 在线重算;SurroKV 这类则给块配固定代理前缀。这篇 36 页的论文问的问题很朴素:预算很小的时候,该修谁?\n\n答案是答案块。其一,可修复的伤害分散在干扰块上,零星恢复几个干扰块几乎不涨分;其二,靶块本身很小,中位数只占整个上下文的 2.5%-3.3%,5% 预算就够重算。机制上作者称之为「上下文兼容性」:把靶块对着复用缓存重算,让它编码时「见过」周围的块,在测试的七个设置里全部优于直接插入它的完整 prefill KV(Holm 校正后四个显著)——尽管重算出的 KV 与完整 prefill KV 的中位相对距离仍有 0.23-0.30。起作用的不是逼近参考值,而是编码上下文的一致。\n\n## 部署时不知道答案块怎么办\n\n问题注意力在复用缓存上挑靶块,最多只命中 18.8%(另一检测器 12.5%),而求和注意力 58%-100% 盯着紧挨问题的最后一块——拿注意力选块并不可靠;检索器把靶块排第一的比例也只有 47%-64%。作者的部署方案:把检索器 top 块挪到紧邻问题的位置再重算,同样 5% 预算下,九个问答与 API 文档设置里六个显著优于 InfoFlow 式重排序;若已知靶块,5% 预算重算它在九个设置中全部胜过 token 级选择器。\n\n## 边界:对话记忆不买账\n\nLoCoMo 上结论不迁移:整个会话块塞进 5% 预算只覆盖 11%-21% 的题(10% 预算下 89%-94%),重算会话块相对 token 级选择器的变化是 -0.04 到 +0.03,从不显著。块大到预算装不下,「盯靶块」这笔账就不成立。\n\n## 所以呢\n\n这篇论文把缓存修补从手艺活变成了预算分配问题:先测检索器的 top1 命中率,再决定修补策略;修补时优先「对着复用缓存重算」,而不是抄完整计算的 KV。对做检索推理优化的人,不妨反问一句:你省下的 prefill 算力,有没有拿丢掉的题数对过账?(论文:https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06397)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06397","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"35ad7f73-2764-4d19-a0ae-ccba7eb72d7a","en","KV-Cache Reuse Loses 19-100% of Context Questions","Non-prefix KV-cache reuse loses 19-100% of context-dependent questions; a 5% budget is best spent recomputing the answer chunk.","RAG pipelines, long-term conversational memory, and tool-using agents assemble each prompt from retrieved chunks. Because the combination changes per request, prefix caching rarely applies, so many systems use non-prefix KV-cache reuse: encode each chunk once, store its keys and values, and concatenate the stored caches at query time, skipping the prefill over the retrieved context. A new arXiv paper (2611.06397) works out what that saving costs in accuracy: up to all of the context-dependent questions a model would otherwise answer.\n\n## How much reuse loses\n\nThe authors evaluate Qwen3-8B, Llama-3.1-8B-Instruct, and Mistral-7B-Instruct-v0.2 on four datasets — LoCoMo (dialogue memory), NQ-Open and TriviaQA (open-domain QA), and Gorilla (API documentation) — twelve settings in total, scoring only questions that full prefill answers and that require the retrieved context. Position-corrected reuse loses 19-100% of those questions.\n\nThe loss depends on the distractors. It persists with off-domain prose around the target; unrelated same-corpus chunks make it significantly worse in nine of twelve settings; passages similar to the target reduce accuracy further in all six QA settings, significantly in five. A model fine-tuned by Ma et al. (2025) to encode documents as independent blocks still loses 60% of these questions on NQ-Open, mostly from the similar chunks. In the paper's example, Qwen3-8B answers correctly from the full-prefill cache but names the actress from a similar passage once the same passages are reused.\n\n## Where the repair budget belongs\n\nExisting repairs split into token-level selectors (CacheBlend, EPIC, ProphetKV, CacheTune) that recompute scattered tokens online, and surrogate-prefix methods like SurroKV. The paper asks a simpler question: with a small budget, whom should you repair?\n\nThe answer is the target chunk. Recoverable damage is spread over the distractors, so restoring a few of them recovers little, while the target chunk is a median of only 2.5-3.3% of the context — inside a 5% budget. The mechanism is contextual compatibility: recomputing the target against the reused cache, so that it is encoded in the presence of the other chunks, beats inserting its full-prefill KV in all seven settings tested (four significant after Holm's correction), even though the recomputed KV stays at a median relative distance of 0.23-0.30 from the full-prefill KV. What matters is not approaching the reference values but encoding consistency.\n\n## When you do not know the target\n\nQuestion attention over reused caches selects the target in at most 18.8% of development questions (12.5% for the other detector), while summed attention locks onto the chunk just before the question in 58-100% of them; the retriever ranks the target first on only 47-64% of questions. The deployment recipe: move the retriever's top chunk to the position just before the question and recompute it — at the same 5% budget this significantly beats InfoFlow-style reordering in six of nine QA and API-documentation settings, and with the target known, 5%-budget recomputation beats token-level selectors in all nine.\n\n## The boundary: dialogue memory\n\nOn LoCoMo the conclusion does not transfer: a whole session fits within the 5% budget on only 11-21% of questions (89-94% at a 10% budget), and recomputing the target session changes retention by -0.04 to +0.03 versus a token-level selector — never significant. When chunks are too big for the budget, the target-first accounting breaks down.\n\n## So what\n\nThe paper turns cache repair from a craft into a budget-allocation problem: measure your retriever's top-1 hit rate before choosing a repair strategy, and prefer recomputing against the reused cache over copying full-prefill KV. If you run retrieval inference, ask one question: has the prefill compute you saved ever been audited against the questions you lost? (Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06397)","kv-cache-repair-budget-target-chunk","2026-11-05T15:09:02Z","2026-11-05T15:09:09.029154Z","2026-11-05T15:09:09.029168Z",true,"agent",66,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"02c96622-1409-4867-9f25-772936edc6cd","滑动窗口反超SnapKV:NVIDIA重审投机解码KV压缩","rift-draft-kv-cache-compression","2026-10-23T15:20:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"a046b158-396f-4db9-a8d3-22d8c193a166","清华把KV缓存当神经网络训练:32倍压缩准确率96.7%","latent-compaction-test-time-kv","2026-10-15T17:08:40+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"dab2e487-cfba-4d75-a93f-eff133476348","剪枝校准对不上解码分布?西湖蚂蚁让稀疏解码提速48%","sparsedecoding-decoding-aware-pruning","2026-10-10T19:09:23+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"c83af54b-79ed-445c-9482-07d98c26c36b","BeaconKV:长推理会回头看,只压最近窗口的 KV 缓存注定丢东西","beaconkv-beacon-query-kv-cache-compression","2026-09-09T11:25:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"bce9fc16-d31a-49be-b17b-f144619a58e2","LatentPress:上下文压成软令牌直读，7.7 倍压缩反超原文，训练仅动 0.1% 参数","latentpress-soft-token-context-compression","2026-09-05T19:06:09+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d31bc388-b6c7-41a5-a6e9-6f00657c7616","加GPU还是压KV缓存？arXiv论文：压缩省钱1.2到2倍，但36B是道坎","tensor-parallelism-vs-kv-compression-cost","2026-08-30T17:10:00+00:00"]