[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-akv-cache-engineering-agents-kv":3,"topics-all":38,"news-related-f2f2f891-a350-4bd1-9055-6eea0c72de66":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"f2f2f891-a350-4bd1-9055-6eea0c72de66","Agent 的 KV 缓存不该由系统单方面接管：清华团队把缓存控制权交回 Agent 侧，并给出 AKV 推理引擎","Agent 轨迹变长后，KV 缓存读写与存储成主导成本，而上下文工程靠裁剪或摘要压体积，会因改动前缀让缓存全部失效并触发重预填充。这篇论文提出「缓存工程」：用 drop 与 repos 算子直接操作活跃缓存不改文本，配合 AKV 引擎，在三个模型、四个基准上平均成本降 27.7%、成绩升 1.7 分。","跑长轨迹的 Agent 会不断调用工具、积累观测，KV 缓存的 I\u002FO 与存储很快变成主要开销。为省成本，主流做法是「上下文工程」——把历史裁剪或摘要压缩。但改动轨迹前缀会让后续缓存全部失效、保留的上下文不得不重新预填充；而为了少付重填的钱把上下文砍得太狠，准确率又会掉。\n\n## 不改文本，直接动缓存\n\n论文的做法是换一层控制面：不修改文本轨迹，而是让 Agent 直接操作活跃的 KV 缓存，并形式化两个算子。\n\n- **drop**：把选中的缓存条目从后续 attention 中排除，K\u002FV 不再参与计算但仍留在系统；\n- **repos**：调整保留下来的 key 的位置编码。配合旋转位置编码（RoPE），repos 把 key 从位置 p 搬到位置 q。\n\n收益很直接：被丢弃条目之外的所有 KV 状态都能复用。这打破了「省成本必然伤准确率」的绑定——不改文本，也就没有前缀失效。\n\n论文还量化了传统做法的代价：只保留最近 12 轮工具响应的裁剪式做法，成本是全上下文基线的 1.85 倍；48k 阈值触发的摘要式做法省约 7% 成本，但准确率掉了 5 个百分点。而直接丢弃较旧的工具响应缓存可复用其余缓存，同时改善准确率与成本。\n\n## 工程上的真实难点：历史依赖\n\n论文点出一个容易被忽略、但对 serving 系统很致命的问题：**历史依赖**。同样一串 token，在不同的缓存操作序列下可能产生不同缓存状态。这意味着传统的 prefix matching 不足以保证正确的缓存复用。\n\nAKV 的系统侧为此协同设计了三样东西：无状态接口、扩展 radix tree、以及 segment-batch attention。对外暴露兼容现有 serving API 的无状态接口，内部管理缓存复用、驱逐与恢复。\n\nAgent 侧用 `CachePolicy` 决定丢什么、何时压实位置，内置按轮次、按 token 数、让模型自行决定三种触发方式。头条配置是「工具响应滚动丢弃」：保留最近 12 个工具响应的 KV 状态。\n\n## 实验怎么做的\n\n评测覆盖 MiniMax-M2.7、GPT-OSS-120B 和 MiroThinker-1.7-Mini；基准覆盖深度检索方向的 BrowseComp-Plus、xbench-DeepSearch-2510、文本版 GAIA 验证子集，以及软件工程方向的 SWE-Verified。Serving 侧用八张 A800-80GB，四张 prefill、四张 decode。\n\n对照实验里，AKV 在三个模型上都取得最高平均成功率，相比两个基线分别提升 1.3–2.1% 与 0.9–8.7%；推理成本相对 Standard 基线降低 27.7%，相对 Text Engineering 基线降低 21.9%。此外让策略在留出任务上递归自我迭代，再降 7.1% 成本。\n\n## 我的看法\n\n这篇论文的价值不在某一个跑分数字，而在它把「缓存由谁管」摆到了正确的高度。此前 KV 缓存基本被视作 serving 系统的内部实现细节；而这篇的论点是：Agent 和 harness 比系统更知道哪些中间步骤没用了。\n\n历史依赖这个发现尤其值得留意。如果不同操作序列会产出不同缓存状态，那么今天很多基于前缀匹配的增量缓存机制，在 Agent 这种高并发、长轨迹的场景下其实并不可靠。这也解释了为什么 AKV 需要定制引擎，而不是在现有系统上加个开关。\n\n如果这条路线成立，Agent harness 与 serving 引擎之间的接口约定，可能成为下一个值得关注的工程标准。仓库 https:\u002F\u002Fgithub.com\u002Fthu-nics\u002FAKV ，论文 https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28577 。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28577","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",{"id":22,"name":23,"slug":23,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"9aceeae0-8ba7-46e7-8cc5-b079ffa23797","en","Cache Engineering: Tsinghua Hands KV Cache Control Back to Agents","Agents operate on KV cache directly via drop and repos operators, avoiding re-prefill. AKV reports 27.7% lower cost and +1.7 points on three models.","An agent working through a long-horizon task keeps calling tools and accumulating observations. As the trajectory grows, KV cache I\u002FO and storage quickly become the dominant cost. The trajectory can even exceed the model's maximum context length, meaning an agent must actively control how its cache grows. The complication is that the mainstream remedy, \"context engineering,\" shrinks that history by trimming or summarizing it. But this creates a structural dilemma: modifying the trajectory prefix invalidates every downstream cached state, so retained context has to be re-prefilled. And trimming context hard enough to avoid re-prefill costs degrades task accuracy.\n\nThe paper puts that dilemma front and center, then runs a controlled comparison across three models and four agent benchmarks, compressing the result into one set of numbers: average benchmark scores improve by 1.7 points, average cost drops by 27.7%, output throughput reaches 1.2x, and P95 request latency sees a 1.5x speedup.\n\n## Changing the cache instead of the text\n\nThe paper's move is to shift the control plane. Rather than rewriting the text trajectory, the agent operates directly on its own active KV cache, formalizing two operators.\n\n- **drop** excludes selected cache entries from future attention, so those keys and values stop participating in computation while remaining in the system.\n- **repos** adjusts the position encodings of retained keys, compacting their position indices so the position sequence stays in logical order, and resets the position index for the next append.\n\nWorking with rotary position embeddings (RoPE), repos moves a cached key from position p to position q via the rotation update K ← R(q)·R(p)⁻¹·K, while cached values remain unchanged.\n\nThe payoff is direct: everything except the dropped entries can be reused without re-prefilling. That breaks the binding between \"saving cost\" and \"hurting accuracy\" that characterizes context engineering — and because the text is never modified, no prefix is invalidated.\n\nThe paper also quantifies the problem with traditional approaches. On the first 100 BCP tasks, with four trials per task and at most 100 agent rounds, trim-based context engineering retaining only the latest 12 rounds of tool responses costs 1.85x the full-context baseline. Summary-based context engineering at a 48k threshold reduces cost by about 7% but lowers accuracy by 5 percentage points. Dropping older tool-response cache directly, by contrast, reuses the remaining cache while improving both accuracy and cost.\n\n## The real engineering obstacle: history dependence\n\nThe paper flags a subtlety that is easy to miss but fatal for serving systems: **history dependence**. The same token sequence can yield different cache states under different cache operation sequences. That means conventional prefix matching is insufficient for correct cache reuse.\n\nAKV's system side co-designs three things in response: a stateless interface, an extended radix tree, and segment-batch attention. Externally it exposes a stateless interface compatible with existing serving APIs, while internally it manages stateful cache reuse, eviction, and restoration.\n\nOn the agent side, a `CachePolicy` decides what to drop and when to compact positions, with three built-in trigger styles: turn-based, token-based, and model-based, where the model decides what to drop and whether to reposition through a `CacheTool`. The system side executes those cache operations through SGLang's OpenAI-compatible Chat Completions API.\n\nThe headline configuration the paper reports is a tool-response rolling-drop policy retaining the KV states of the latest 12 tool responses, with its repos threshold matched to each baseline's summarization threshold.\n\n## How the experiments were run\n\nAt the model level, evaluation covers three LLMs: MiniMax-M2.7, GPT-OSS-120B, and MiroThinker-1.7-Mini. The benchmarks span four agent tasks: BrowseComp-Plus (BCP), xbench-DeepSearch-2510 (xbench-DS), the text-only 103-task GAIA validation subset (GAIA-Text) for deep research, and SWE-Verified for software engineering.\n\nOn the serving side, baselines run on SGLang, which is also AKV's base serving system, using eight NVIDIA A800-80GB GPUs — four for prefill and four for decode, with tensor parallelism of 4. Because the A800 lacks native MXFP4 support, all LLM weights are set to BF16.\n\nIn the controlled comparison, AKV achieves the highest average success rate across all three models, improving the average success rate by 1.3–2.1% and 0.9–8.7% against two baselines respectively. Inference cost falls by 27.7% relative to Standard and 21.9% relative to Text Engineering, averaged across all three models and four benchmarks. Letting the policy iterate recursively on held-out tasks reduces cost by a further 7.1% compared with expert-designed policies.\n\n## My read\n\nThe value of this paper is not any single benchmark number but the way it frames \"who owns the cache.\" KV cache was previously treated as an internal implementation detail of the serving system — the system decides what to reuse and what to evict. This paper argues that the agent and its harness actually know better than the system which intermediate steps have become useless, and therefore should be able to issue cache operations directly, with the system executing them efficiently.\n\nThe history-dependence finding deserves particular attention. If different cache operation sequences can produce different cache states, then many incremental-caching mechanisms built on prefix matching are unreliable in exactly the high-concurrency, heavily branched, long-trajectory settings agents create. AKV treats it as a first-class citizen, which also explains why it needs a purpose-built engine rather than a configuration flag on an existing system.\n\nIf that framing holds, the interface contract between agent harnesses and serving engines may become an engineering standard worth watching. Both paper and code are public: the repository is at https:\u002F\u002Fgithub.com\u002Fthu-nics\u002FAKV and the paper at https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.28577 .","akv-cache-engineering-agents-kv","2026-10-22T07:09:11Z","2026-10-22T07:21:45.877757Z","2026-10-22T07:21:45.877765Z",true,"agent",29,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"a820bb5a-ac48-4aab-8d8b-34da340900c1","REMORY 让 LLM 用 5% 上下文继续跑 Agent","remory-agent-memory","2026-10-11T09:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"bc642ebf-9b1c-41cf-98d1-dd55393fd429","HeadWiseKV:无训练KV cache压缩让混合LLM长上下文从114K推到161K","headwisekv-training-free-kv-cache-compression","2026-09-03T03:44:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"f65e204c-0115-4b50-9113-2c3bb2ff6637","ReCache:给 Agent 的工具记忆装上独立缓存,KV 内存砍 92%、首 token 提速 3.655 倍","recache-agent-kv-cache-reuse","2026-08-24T15:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"822987d9-faaa-492d-b7ba-1c0427f72f9b","AgentOps 出海第一步：拆腾讯云 ADP 4.0 海外版的「Agent+Workflow 分账」架构","tencent-adp-4-0-overseas-agentops","2026-07-20T02:11:09+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"d016b7c3-4fbe-4b30-871a-f0d503f62228","DuoMem 用「双空间蒸馏」把 4B 端侧 Agent 拉到 72B 教师水位:ALFWorld 任务率 4.3% → 77.9%","duomem-4b-on-device-agent","2026-07-05T18:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"b35a8c22-9a47-412c-88bd-1e1400bdba98","Perplexity「Search as Code」：把搜索 API 拆成可编程原语，让 AI Agent 自己写检索管线","perplexity-search-as-code-100pct-cve-85pct-token","2026-06-08T16:00:00+00:00"]