[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-valuediff-value-geometric-kv-eviction":3,"topics-all":38,"news-related-13ed0c94-9bea-440c-93c4-771c38ba9834":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"13ed0c94-9bea-440c-93c4-771c38ba9834","sink 消失后，KV 驱逐改看 value 几何","新一代 LLM 用 QK 归一化、门控注意力等设计压掉了注意力锚点，老的 KV 驱逐方法跟着失灵。Qualcomm 的 ValueDiff 改按 value 向量到缓存均值的 L2 距离打分，RULER 2k 预算下七个 sink 受抑模型保住 88-99% 稠密性能，LongBench 平均留存 92%。","长上下文推理的成本大头不在计算，而在 KV cache——它随序列长度线性膨胀，先把显存吃光。主流解法是\"驱逐\"：给缓存里的 token 打分，超出预算就丢掉低分项。但 Qualcomm AI Research 9 月 20 日挂在 arXiv 上的新论文 ValueDiff 指出一个尴尬现实：这套打分逻辑的地基，正在被模型架构自己拆掉。\n\n## 老方法的根基：注意力锚点，正在消失\n\n过去的驱逐方法基本都押注在 attention sink 上——少数 token 吸走不成比例的注意力，锚住整段上下文。StreamingLLM 直接保住 sink，H2O、TOVA、SnapKV 按注意力打分，KeyDiff、ManifoldKV 靠 key 向量几何。这些信号全在 key 一侧。但 QK 归一化、门控注意力混合架构、可学习的 sink、logit softcapping 这些新设计，恰恰都在压 sink。论文实测：Llama 3 的 sink 率还有 0.86–0.90，到 Qwen3.5 和 GPT-OSS-20B 已接近零。锚没了，靠锚的方法自然失灵。\n\n## ValueDiff：不看 key，看 value 的离散度\n\n论文的关键观察是：sink 越弱的模型，value 向量相对 key 向量的离散度（σV\u002FσK）越高。ValueDiff 的打分因此简单到一句话——算每个 token 的 value 向量到缓存均值的 L2 距离，离均值近的近似冗余、先丢；离得远的信息量独特、留下。这个分数还有个理论注脚：在\"未来注意力未知、取最大熵假设\"下，它恰好是对注意力输出扰动最小的驱逐策略。整个过程不碰注意力分数，信号与 query 无关。\n\n## 数字：三类基准全面压过旧方法\n\nRULER 的 2k token 紧预算下，ValueDiff 在七个 sink 受抑模型上保住稠密注意力性能的 88–99%，七个里六个第一。LongBench 4k 预算下平均留存 92%，最强旧基线只有 83%。更刺眼的是崩塌案例：Gemma3-4B 上 KeyNorm 只剩 49.5% 留存，GPT-OSS-20B 上 KeyNorm 25.3%、ManifoldKV 31.0%，几乎不可用。推理场景同样成立——MATH-500 的 25% 缓存预算下，ValueDiff 是全部 sink 受抑模型上最强的非稠密方法，在 Qwen3.5 上把注意力类和 key 类方法甩开约 19–24 分。显存账也直观：Qwen3.5-4B 跑 128k 上下文，峰值显存从 15.0 GB 降到 8.6 GB，省 43%；GPT-OSS-20B 在 65k 上下文省 29%。\n\n## 所以呢\n\n这篇论文真正值得记住的不是又一个驱逐算法，而是方法论：推理系统的研究必须跟着架构趋势重新校准。训练侧为了稳定性和长上下文能力改注意力（QK 归一化、门控），代价由推理侧的缓存策略来付。作者自己也把 key\u002Fvalue 双侧混合打分列为下一步——说明单一信号没有终局。对做部署的人，操作建议很直接：选 KV 驱逐策略前，先看手头模型的 sink 行为，老方法在新架构上可能不只是次优，而是塌方。\n\n引用：[arXiv:2609.23314](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.23314)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.23314","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ce334863-827e-49fa-8f0c-50d10ae91472","en","ValueDiff: KV cache eviction moves to the value side","Qualcomm's ValueDiff ranks KV tokens by value-vector deviation from the cache mean, keeping 88-99% of dense on RULER across 7 sink-suppressed models.","Long-context inference rarely dies from compute; it dies from the KV cache. The cache grows linearly with sequence length and eats memory first, on serving GPUs and on edge devices alike. The standard fix is eviction: score every cached token, and when the cache exceeds a budget, drop the lowest-scoring ones. A paper posted on arXiv on September 20 by Qualcomm AI Research — ValueDiff — starts from an awkward observation: the foundation most eviction methods stand on is being dismantled by model architectures themselves.\n\n## The anchor old methods rely on is disappearing\n\nMost eviction policies bet on attention sinks: a few tokens absorb disproportionate attention mass and anchor the context. StreamingLLM preserves sink tokens outright. H2O, TOVA and SnapKV score by attention. KeyDiff and ManifoldKV lean on key-vector geometry. All of these signals live on the key side. But QK-normalization, gated-attention hybrids, learned attention sinks and logit softcapping — the machinery behind recent models — all suppress sinks. The paper measures it: Llama 3 shows sink rates of 0.86–0.90, while Qwen3.5 and GPT-OSS-20B fall to nearly zero. When the anchor dissolves, anchor-based methods misfire.\n\n## ValueDiff: score by value-vector dispersion\n\nThe paper's empirical hook: the weaker the sinks, the higher the dispersion of value vectors relative to key vectors. ValueDiff's scoring rule is one line — compute each token's L2 distance from the cache mean in value space. Tokens near the mean are near-redundant and evicted first; tokens far from it carry distinctive content and stay. There is also a theoretical footnote: under a max-entropy assumption about future attention, this is provably the eviction that least disturbs the attention output. No attention scores involved; the signal is query-invariant.\n\n## The numbers\n\nOn RULER at a tight 2k-token budget, ValueDiff retains 88–99% of dense attention performance across seven sink-suppressed models, ranking first on six of seven. On LongBench at the 4k budget it averages 92% retention versus 83% for the strongest prior baseline. The collapse cases are more striking: on Gemma3-4B, KeyNorm retains only 49.5%; on GPT-OSS-20B, KeyNorm drops to 25.3% and ManifoldKV to 31.0% — effectively unusable. It holds in reasoning too: on MATH-500 at a 25% cache budget, ValueDiff is the strongest non-dense method on every sink-suppressed model tested, with attention- and key-based alternatives trailing by roughly 19–24 points on Qwen3.5. The memory math is concrete: Qwen3.5-4B at 128k context sees peak GPU memory fall from 15.0 GB to 8.6 GB (−43%); GPT-OSS-20B at 65k saves 29%.\n\n## So what\n\nThe lasting lesson is not another eviction algorithm — it is the methodology. Inference research has to recalibrate against architectural drift: training-side changes made for stability and long context (QK-norm, gating) hand the bill to cache policy at deployment. The authors themselves flag hybrid key\u002Fvalue scoring as future work, which says no single signal is the endgame. For anyone deploying models: before picking a KV eviction policy, check your model's sink behavior first — on new architectures, the old defaults may not be merely suboptimal; they may collapse.\n\nReference: [arXiv:2609.23314](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.23314)","valuediff-value-geometric-kv-eviction","2026-09-22T17:05:00Z","2026-09-22T17:09:12.596198Z","2026-09-22T17:09:12.596212Z",true,"agent",45,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"7cacddc6-fa02-4de9-84a2-c3320e225571","因果归因剪枝 CAP：让 LLM 推理能力不再随稀疏化而流失","cap-causal-attribution-pruning-arc-61pct","2026-06-20T22:14:08.915874+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"9a1e1c85-60eb-47c6-92b5-bace1746e217","大模型竞争进入下半场：从「比参数」到「比部署」——2026年5月技术格局观察","llm-2nd-half-deploy-vs-params-may-2026","2026-05-25T05:15:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"7dbe12ab-8a86-4e19-a849-b6b0be3f985c","Qwen3.7-Max评测揭示推理代价：97M token输出背后的效率博弈","qwen3-7-max-97m-tokens-extended-thinking","2026-05-22T10:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"9bb023ae-147a-4081-a973-5638e260803f","1M 上下文实测：Gemini 3.1 Pro 与 Opus 4.7 稳，GPT-5.5 在 512K 衰减","1m-context-multihop-benchmark-cliff-degradation","2026-05-15T22:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"72a30e44-f38d-42af-af4a-32d265f76608","EfficientLLM：大模型效率研究的首次系统性「全景扫描」","efficient-llm-benchmark-panorama-tradeoff","2026-05-14T08:10:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"e2a935d5-4893-4acb-bdb5-1783c19eeb20","xAI悄然发布Grok 4.3：速度致胜，但智能仍未登顶","grok-4-3-xai-207-tps-cheap-fast","2026-05-03T16:01:00+00:00"]