[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-beaconkv-beacon-query-kv-cache-compression":3,"topics-all":38,"news-related-c83af54b-79ed-445c-9482-07d98c26c36b":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"c83af54b-79ed-445c-9482-07d98c26c36b","BeaconKV:长推理会回头看,只压最近窗口的 KV 缓存注定丢东西","ICML 2026 论文提出 BeaconKV:用少量信标 query 预判哪些 KV 会被回看,训练无关。峰值显存最多降到约 1\u002F5.8,吞吐提升 4.3 倍以上,精度较现有方法最多高 31.7 个百分点。","大模型做长链推理时,吃不消的往往不是算力而是显存。Chain-of-Thought 越写越长,KV 缓存随序列长度线性膨胀,一条几万 token 的轨迹就能把 GPU 顶穿。常规解法是压缩 KV 缓存:给历史 token 打分,留高分扔低分。但打分依据是什么?几乎所有方法都默认同一条假设——**最近几步的 query 可以代表未来的注意力模式**。汉阳大学与成均馆大学的团队在一篇 ICML 2026 论文里把这条假设证伪了。\n\n## 长推理里有\"回头看\"的时刻\n\n论文的观察起点是一类被命名为 Thought Revisiting Tokens(TRT,思路回溯 token)的解码步骤。模型在长推理中并非一路向前:某些时刻它会突然回头,重新关注推理轨迹早期形成的任务规划,以维持全局连贯。\n\n问题就在这里。RPC、R-KV 这类为长推理设计的压缩方法,都用最近若干步 query 估计哪些 KV 该留。可 TRT 的落点不在最近窗口视野里——等模型真要回头找早期规划时,那部分 KV 已被当作低分内容驱逐。显存省了,推理链条却断了。\n\n作者进一步做几何分析,发现 TRT 对应的 query 并非散乱分布,而是在嵌入空间里**聚成少数几个相似性簇**。既然是簇,就能用少量代表点概括。\n\n## 用信标 query 预判\"哪些会被回看\"\n\nBeaconKV 由此而来:维护一小组 beacon queries(信标 query)作为各全局 query 簇的紧凑代表,预判哪些 KV 未来会被重新访问,不必存下整段 query 历史。机制三步:从已生成轨迹采样出信标 query,解码中通过 Continual FPS(持续最远点采样)在线刷新;缓存超预算时,用信标 query 连同最近 query 的注意力共同给历史 KV 打分;只保留最高分条目,同时永远保住一个最近滑动窗口。方法**不需要训练**,是纯推理期的插入式改造。\n\n## 数字:精度差距最大 31.7 个百分点\n\n实验覆盖四个开源长推理模型(R1-Distill-Qwen-7B、R1-Distill-Llama-8B、Qwen3-4B、Qwen3-14B)和四个基准(AIME24、MATH-500、GPQA-Diamond、LiveCodeBench)。论文报告:精度较现有压缩方法最多高 31.7 个百分点;激进压缩下峰值显存最多降到约 1\u002F5.8,吞吐相对未压缩基线提升 4.3 倍以上。\n\n效率表更说明取舍。单张 A100 80GB、Qwen3-4B、32K 生成长度下,Full KV 在 batch 14 时吞吐 82.3 tokens\u002Fs、显存 77.0 GB、LiveCodeBench 精度 54.4;BeaconKV 在 2K 预算同 batch 下为 356.4 tokens\u002Fs、13.3 GB、51.1。和 RPC 对比:1K 预算、batch 320 时吞吐(1380.8 对 1345.9)与显存(72.0 对 72.5 GB)几乎打平,精度却是 29.9 对 42.2——不是靠多花资源换精度,而是**在同样预算里把该保住的内容留对了**。\n\n## 所以呢\n\n这篇的价值不在\"又快了几倍\",而在指出一处方法论错位:**长推理模型边生成边产生上下文,和传统长上下文处理不是同一个问题**。后者上下文给定,重要性可静态估计;前者由模型自己写出上下文,且会回头读自己写过的东西。用最近窗口猜未来注意力,先天不成立。\n\n压缩的天花板不取决于打分函数多精巧,而取决于观察窗口是否符合模型真实的注意力动力学。代码已开源([arXiv:2609.04971](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.04971))。\n\n模型会回头看自己写下的推理链——这件事本身,比任何加速倍数都更值得琢磨。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.04971","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"c89957ee-249f-4e7f-960b-dd20f28580e2","en","BeaconKV: Beacon Queries Fix KV Eviction for Long Reasoning","An ICML 2026 paper keeps a few beacon queries to predict which KV entries get revisited: 5.8x less memory, 4.3x throughput, +31.7 points over prior work.","When large language models run long chains of thought, the binding constraint is usually not compute but memory. The key-value cache grows linearly with sequence length, and a reasoning trace of tens of thousands of tokens can exhaust GPU memory outright. The standard remedy is KV cache compression: score past tokens, keep the important ones, evict the rest. But what should the score be based on? Nearly every existing method leans on the same assumption — that **queries from the most recent decoding steps are a reliable proxy for future attention patterns**.\n\nA team from Hanyang University and Sungkyunkwan University has now falsified that assumption, in a paper accepted to ICML 2026.\n\n## Long reasoning has moments of looking back\n\nThe paper starts from a phenomenon the authors call Thought Revisiting Tokens (TRT). Models doing long-horizon reasoning do not move strictly forward: at certain decoding steps they re-attend to distant earlier context — typically the task-solving plan formulated early in the trace — in order to maintain global coherence. The attention targets of these queries sit far behind the current position.\n\nThat is precisely where recent-query scoring breaks. RPC and R-KV, both designed for reasoning models, estimate which KV entries matter using only the last few queries. TRT targets are invisible to that window, so by the time the model actually wants to revisit its early plan, those entries have already been evicted as low-scoring. Memory is saved; the reasoning chain is broken.\n\nA geometric analysis surfaces something more useful: TRT queries are not scattered. In embedding space they **cluster into a small number of similarity groups** — and clusters can be summarized by a few representatives.\n\n## Beacon queries anticipate what will be revisited\n\nHence BeaconKV: maintain a compact set of beacon queries as representatives of each global query cluster, and use them to anticipate which KV pairs will be revisited, without storing the entire query history.\n\nThe mechanism has three parts. A small set of beacon queries is sampled from the generated reasoning trace and refreshed on the fly during decoding via Continual Farthest Point Sampling. When the decode-time KV cache exceeds its budget, past entries are scored using attention from these beacon queries together with the most recent queries. Only the highest-scoring entries survive, while a recent sliding window is always preserved. Sampling operates in the pre-RoPE query space to capture geometric diversity.\n\nThe method is **training-free** — a pure inference-time modification. The open-source implementation defaults to 16 beacon queries, 16 recent queries, and a window size of 32.\n\n## The numbers: up to 31.7 points of accuracy gap\n\nExperiments span four open-source reasoning models (R1-Distill-Qwen-7B, R1-Distill-Llama-8B, Qwen3-4B, Qwen3-14B) across AIME24, MATH-500, GPQA-Diamond and LiveCodeBench. The paper reports accuracy gains of up to 31.7 percentage points over existing compression methods, peak GPU memory reduced by up to 5.8x under aggressive compression, and throughput improvements of over 4.3x relative to the uncompressed baseline.\n\nThe efficiency table shows the trade-off concretely. On a single A100 80GB running Qwen3-4B with 32K generation length, Full KV at batch size 14 delivers 82.3 tokens\u002Fs, 77.0 GB peak memory and 54.4 LiveCodeBench accuracy. BeaconKV at a 2K budget and the same batch size gives 356.4 tokens\u002Fs, 13.3 GB and 51.1 accuracy — roughly one sixth the memory for 3.3 points of accuracy.\n\nThe head-to-head with RPC is more telling. At a 1K budget and batch size 320, throughput (1380.8 vs 1345.9) and memory (72.0 vs 72.5 GB) are essentially tied, but accuracy is 29.9 versus 42.2. BeaconKV is not buying accuracy with extra resources; it is **keeping the right things inside the same budget**.\n\nOne ablation is counterintuitive. On Qwen3-4B \u002F AIME24 with a 2048 budget, more beacons is not better: the (4 beacon, 28 recent) configuration reaches the highest accuracy at 66.5, while the all-recent RPC configuration scores 51.3. A handful of beacons produces a double-digit gap, which suggests what is being captured is structure rather than volume.\n\n## So what\n\nThe contribution here is less \"another speedup\" than a methodological correction: **a reasoning model generates its own context as it goes, which makes it a fundamentally different problem from conventional long-context processing**. In the latter, the context is given and importance can be estimated statically. In the former, the model writes the context and then reads back what it wrote. Using a recent window as a proxy for future attention is unsound by construction in that setting.\n\nInference cost is becoming the dominant line item in deploying large models, and KV cache compression has drawn a dense stream of papers lately. BeaconKV's reminder is that the ceiling on compression depends less on how clever your scoring function is than on whether your observation window matches the model's actual attention dynamics. Code is open source ([arXiv:2609.04971](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.04971); implementation at aiha-lab\u002FBeaconKV).\n\nModels look back at the reasoning they have written. That fact may be worth more attention than any speedup multiple.\n","beaconkv-beacon-query-kv-cache-compression","2026-09-09T11:25:00Z","2026-09-09T11:17:42.537865Z","2026-09-09T11:17:42.537875Z",true,"agent",181,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"bce9fc16-d31a-49be-b17b-f144619a58e2","LatentPress:上下文压成软令牌直读，7.7 倍压缩反超原文，训练仅动 0.1% 参数","latentpress-soft-token-context-compression","2026-09-05T19:06:09+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"d31bc388-b6c7-41a5-a6e9-6f00657c7616","加GPU还是压KV缓存？arXiv论文：压缩省钱1.2到2倍，但36B是道坎","tensor-parallelism-vs-kv-compression-cost","2026-08-30T17:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"d2c56430-f9a7-4844-bfab-a8651279a70c","ResKV 不再把 KV 缓存压缩等同于删词：给被淘汰的信息留一份残差账本","reskv-residual-kv-cache-compression","2026-08-03T10:43:23+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"5f745fe5-ea5d-453a-8b08-7dac524d1ac2","ACL 2026 综述 sKis：KV 缓存优化重塑为 LLM serving 系统学","acl-2026-skis-kv-cache","2026-07-12T18:15:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"dd9d199c-5cd9-4a15-8ae6-7e4fb40f4129","MosaicKV:把 KV 缓存压成「马赛克」,长上下文推理跑出 16× 注意力加速","mosaickv-mosaic-compression","2026-07-03T18:01:00+00:00"]