[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-recache-agent-kv-cache-reuse":3,"news-related-f65e204c-0115-4b50-9113-2c3bb2ff6637":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"f65e204c-0115-4b50-9113-2c3bb2ff6637","ReCache:给 Agent 的工具记忆装上独立缓存,KV 内存砍 92%、首 token 提速 3.655 倍","Agent 推理有个被忽视的浪费:工具 schema 在每次请求中以不同组合重复编码,标准前缀缓存完全失效。arXiv 新论文 ReCache 用\"资源级注意力\"生成组合不变的 KV 块,再配路由与剪枝,KV 内存砍 92.43%,首 token 提速 3.655 倍,调用准确率几乎无损,代码已开源。","Agent 每接一个新请求,都可能在重新\"背\"一遍它已经背过无数次的工具说明书。arXiv 8 月 20 日提交的论文 ReCache 指出:工具和技能的 schema 会以不同组合、不同顺序在请求之间反复出现,而标准前缀缓存要求前缀完全一致才能命中,这套机制在 Agent 场景下几乎形同虚设。\n\n## 问题:前缀缓存为什么对 Agent 失效\n\n传统 prefix caching 的复用条件苛刻:只有当两段上下文从头到尾逐 token 一致时,KV 状态才能直接复用。但 Agent 的实际运行方式是——同一批工具,这次按 A、B、C 的顺序拼进 prompt,下次变成 C、A、B,再下次混进两个新技能。组合一变,前缀断裂,缓存全部作废,模型只能把每个 schema 从头再编码一遍。论文将其定性为纯粹的重复计算:同一段工具描述,可能在服务集群里被编码了成千上万遍。\n\n## 方案:让 KV 块\"组合不变\"\n\nReCache 的核心思路是把每个资源(schema)当成独立单元来缓存,而不是整条 prompt 的附属品,分三层落地:\n\n- **资源级注意力(Resource-wise attention)**:切断资源之间的交叉交互,并给每个资源分配局部位置编码。这样无论资源以什么组合、什么顺序出现,它编码出的 KV 块都保持一致——论文称之为\"组合不变性\"。这是独立缓存得以成立的前提。\n- **贡献路由**:把资源的可见性限制在按贡献筛选出的\"层-KV 头组\"通路上,不是所有层、所有头都需要看到所有资源。\n- **结构+语义双剪枝**:只保留调用真正需要的字段,工具描述里的冗余部分在缓存阶段就被裁掉。\n\n## 效果:三个关键数字\n\n论文在由七个公开工具\u002F技能调用数据集组装的基准上评估(含资源不相交测试):\n\n- 资源级注意力单独使用,调用性能与稠密注意力几乎打平:82.3% vs 82.4% Inv-F1,同时换来 3.655 倍的首 token(TTFT)加速;\n- 完整框架把分配的 KV 张量内存砍掉 92.43%;\n- 注意力计算本身提速 1.423 倍。\n\n换句话说,用约 0.1 个百分点的调用准确率代价,换回内存与延迟的大幅下降。\n\n## 怎么看\n\n这篇论文的价值不在单项数字,而在它点破了一个行业惯性:大家默认缓存是\"前缀\"问题,但 Agent 时代资源是动态组合的,缓存粒度必须下沉到资源级。它与社区里非前缀 KV 复用、KV 量化压缩的路线一脉相承,但\"组合不变 KV 块\"这个设计把复用条件从\"前缀一致\"放松到\"资源一致\",更贴近 Agent 的真实调用模式。对做 Agent 推理服务的团队,这是值得对照自身缓存命中率读的一篇:如果你的 Agent 业务前缀命中率常年上不去,问题可能不在缓存实现,而在缓存模型本身。代码已开源(github.com\u002FEIT-NLP\u002FReCache),论文原文:https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.19662","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.19662","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"f6428eab-8004-49ca-8faa-c02ff0728080","en","ReCache: Independent Caching for Agent Tool Memory — 92% KV Memory Cut, 3.655x Faster First Token","Agent inference has an overlooked waste: tool schemas get re-encoded in different combinations on every request, defeating standard prefix caching. The new arXiv paper ReCache uses resource-wise attention to produce composition-invariant KV blocks, plus routing and pruning, cutting KV memory by 92.43% and speeding up time-to-first-token by 3.655x with nearly no loss in invocation accuracy. Code is open-sourced.","Every time an agent picks up a new request, it may be re-memorizing tool documentation it has already seen thousands of times. A paper submitted to arXiv on August 20, titled ReCache, points out that tool and skill schemas recur across requests in different combinations and orders, while standard prefix caching only hits when prefixes match exactly — making the mechanism nearly useless for agents.\n\n## Why Prefix Caching Fails for Agents\n\nTraditional prefix caching has strict reuse conditions: KV states can only be reused when two contexts are token-for-token identical from the start. But agents actually run differently — the same set of tools gets assembled into the prompt as A, B, C this time, C, A, B next time, plus two new skills the time after. Once the combination shifts, the prefix breaks, the cache is invalidated, and the model re-encodes every schema from scratch. The paper frames this as pure redundant computation: the same tool description may be encoded countless times across a serving cluster.\n\n## The Approach: Composition-Invariant KV Blocks\n\nReCache's core idea is to cache each resource (schema) as an independent unit rather than as an appendage of a whole prompt. It lands in three layers:\n\n- **Resource-wise attention**: removes cross-resource interactions and assigns resource-local positions, so a resource's KV block stays identical no matter which combination or order it appears in — the paper calls this composition invariance. This is the precondition that makes independent caching possible.\n- **Contribution-selected routing**: restricts a resource's visibility to contribution-selected layer–KV-head-group routes. Not every layer and every head needs to see every resource.\n- **Structural and semantic pruning**: retains only invocation-critical fields, trimming redundant parts of tool descriptions at the caching stage.\n\n## Results: Three Key Numbers\n\nThe paper evaluates on a benchmark assembled from seven public tool- and skill-use datasets, including resource-disjoint tests:\n\n- Resource-wise attention alone matches dense invocation performance at 82.3% versus 82.4% Inv-F1, while delivering a 3.655x time-to-first-token speedup;\n- The complete framework reduces allocated KV-tensor memory by 92.43%;\n- Attention computation itself accelerates by 1.423x.\n\nIn other words, roughly 0.1 percentage points of invocation accuracy trades for massive reductions in memory and latency.\n\n## Our Read\n\nThe paper's value lies not in any single number but in puncturing an industry assumption: everyone treats caching as a \"prefix\" problem, but in the agent era resources combine dynamically, so cache granularity must sink to the resource level. It sits in the same lineage as non-prefix KV reuse and KV quantization work in the community, yet the \"composition-invariant KV block\" design relaxes the reuse condition from \"prefix match\" to \"resource match,\" which fits real agent invocation patterns. For teams running agent inference services, this is a paper worth reading against your own cache hit rates: if your agent business chronically shows low prefix hit rates, the problem may not be your cache implementation but the caching model itself. Code is open-sourced at github.com\u002FEIT-NLP\u002FReCache; paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.19662","recache-agent-kv-cache-reuse","2026-08-24T15:30:00Z","2026-08-23T21:11:52.309604Z","2026-08-23T21:11:52.309626Z",true,"agent",59,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"822987d9-faaa-492d-b7ba-1c0427f72f9b","AgentOps 出海第一步：拆腾讯云 ADP 4.0 海外版的「Agent+Workflow 分账」架构","tencent-adp-4-0-overseas-agentops","2026-07-20T02:11:09+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"d016b7c3-4fbe-4b30-871a-f0d503f62228","DuoMem 用「双空间蒸馏」把 4B 端侧 Agent 拉到 72B 教师水位:ALFWorld 任务率 4.3% → 77.9%","duomem-4b-on-device-agent","2026-07-05T18:00:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"5878a668-282c-4b88-b2b8-7eef40b7938c","LFM2.5-2.6B：2.5GB 内存跑本机 Agent 220 tok\u002Fs","lfm2-5-2-6b-on-device-agent","2026-08-11T00:00:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"15987a0e-bc06-4f21-9608-264da02d0e6c","AutoMem 让 32B 开源模型在长程任务上追平 Claude Opus 4.5","automem-stanford-32b-long-horizon","2026-07-23T12:10:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"6784d3bd-26c1-4fd5-a2e5-8c7b9e591dae","SmoothAgent 把上下文变换「提前做」：Agent 长链路 TTFT 砍到原来的 1\u002F12","smoothagent-ttft-12x","2026-07-23T03:00:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"c6dc2edc-1a4a-46b9-85e6-f1c8ef32faa6","DeepSeek DSpark 跑进 Apple Silicon：mlx-dspark 给出首个原生 MLX 移植,逐字节保持原模型输出","mlx-dspark-apple-silicon","2026-07-04T12:00:00+00:00"]