[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-declarative-attention-kv-cache-declare":3,"news-related-b4f270b3-db43-4586-a0e5-a062320c6d1b":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"b4f270b3-db43-4586-a0e5-a062320c6d1b","让模型自己声明看哪里:Declarative Attention 零训练砍 52% KV 读取","KAIST AI 与 Google DeepMind 提出 Declarative Attention:模型在思维链里用三种标签声明注意力范围,推理引擎据此跳过大部分 KV cache 读取。15 项长上下文任务零样本测试中,Gemma-4-31B 解码读取砍 52.0%,精度仅降 1.27 个百分点。","在一场百万 token 的对话里,用户只问了一句「第三章那家公司哪年成立」——全局注意力层却要为回复的每一个 token 把整份 KV cache 从头读到尾。长上下文推理的瓶颈不在参数量,而在这种每步一次的全量访存。KAIST AI 与 Google DeepMind 9 月 2 日提交的论文(arXiv:2609.02737)提出 Declarative Attention(DA),思路反直觉:与其用外部评分器预测哪些 token 重要,不如让模型在思维链里直接声明「我现在只需要看哪一段」。\n\n## 协议:三种注意力模式,像工具调用一样解析\n\nDA 把生成分成三个模式:`\u003Cglobal>` 读全上下文用来导航,`\u003Cfocus>` 只读点名的片段,`\u003Clocal>` 完全不看上下文、只基于已生成的回复做自包含推理。长输入切成约 2048 token 的「magic chunks」,推理引擎的状态机像解析工具调用一样解析声明,直接改写 vLLM 的 KV cache block 表,让注意力内核少读。实现不动 kernel 和调度器,只在 attention metadata builder 挂钩子;掩码按 block(16-32 token)对齐,只作用于全局注意力层——滑动窗口与线性注意力层不随上下文增长,无需省。\n\n方向还可逆:DA 只掩码、不驱逐,cache 全量驻留,`\u003Cfocus>` 之后随时切回 `\u003Cglobal>` 重新全量读——这是它保住精度的结构原因。\n\n## 数字:零训练砍 52% 读取,精度掉 1 个点\n\n在 15 项长上下文任务(RULER、LongBench v1\u002Fv2、LooGLE、ZeroScrolls)上对六个现成模型(Gemma-4-31B\u002F12B\u002FE4B、Qwen-3.6-27B、Qwen-3.5-9B\u002F4B)零样本评测:Gemma-4-31B 解码期间总 attended token 减少 52.0%,平均精度只掉 1.27 个百分点(87.01% 降到 85.74%);Qwen-3.6-27B 减少 31.1%,精度掉 2.75 个百分点。差距随规模从 4B 到 31B 收窄,单次回复最多省下 2100 万个 attended token。消融显示动态掩码贡献大头:相对无掩码版最多再砍 71.1%。\n\n作者用 roofline 模型(B200、bf16、70% 带宽利用率)推算:优化服务栈上解码墙钟时间降到 0.71 倍(Gemma)与 0.77 倍(Qwen)。代价是解码步数变多(+35%\u002F+31%),但全局读取已占 vanilla 解码时间 73%(Gemma)与 86%(Qwen),少读比少步划算。\n\n## 与现有路线的差别\n\n主流稀疏注意力靠索引器或代理评分预选 token,每步仍付 O(N) 扫描;解析一个 `\u003Cfocus>` 标签是 O(1)。驱逐式压缩直接扔 KV 条目,DA 保留全量 cache、掩码每步可逆。作者称之为稀疏注意力的「新轴」——控制权从系统侧挪进模型自己的输出流。\n\n## 冷水也得泼\n\n论文自认两条硬限制:其一,所有测试模型在 thinking 模式下都无法遵循 DA 协议,实验全部禁用思考模式——与当前推理模型主流路线正面摩擦;其二,零样本结果只是下界,协议尚未经后训练进权重。roofline 的 0.71 倍是建模推算而非实测,1M 语境下「全局读取占解码 98.53%」这类占比也随上下文长度快速变化。\n\n所以呢:DA 最值钱的不是那 52%,而是验证了模型「知道自己在看什么」可以外化成可解析的文本协议。后训练若把这套声明训进权重,而 vLLM 只改 block 表就能接入,长上下文推理成本账本可能真要重算。\n\n参考:arxiv.org\u002Fabs\u002F2609.02737","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.02737","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"57394798-7466-483c-8603-3c1e807dacf4","en","Models can now declare their own attention: 52% fewer KV reads","Declarative Attention: models declare attention scope in CoT tags; engine skips KV reads. Gemma-4-31B cuts attended tokens 52.0% at 1.27pp cost.","In a million-token conversation, the user asks about one detail from chapter three — yet global attention layers must scan the entire KV cache to generate every single token of the reply. The bottleneck of long-context inference is not parameter count but this per-step full-context memory traffic. A paper submitted on Sep 2 by KAIST AI and Google DeepMind (arXiv:2609.02737) proposes Declarative Attention (DA), with a counterintuitive idea: instead of using external scorers to predict which tokens matter, let the model itself declare, inside its own chain-of-thought, which region it needs to read.\n\n## The protocol: three attention modes, parsed like tool calls\n\nDA partitions generation into three modes: `\u003Cglobal>` reads the full context for navigation, `\u003Cfocus>` reads only the named segments, and `\u003Clocal>` reads no context at all, doing self-contained reasoning over the response so far. The long input is split into \"magic chunks\" of roughly 2048 tokens, and the inference engine runs a state machine that parses these declarations — exactly like parsing tool calls — then rewrites the KV-cache block table in vLLM so the attention kernel simply reads less. No kernel modifications, no scheduler changes: the integration is a hook on the attention metadata builder. The mask is aligned to blocks (16-32 tokens) and applies only to global attention layers; sliding-window and linear-attention layers do not grow with context, so there is nothing to save there.\n\nThe key property is reversibility: DA masks but never evicts. The cache stays fully resident, and a `\u003Cfocus>` step can be followed by a `\u003Cglobal>` step that re-attends everything — the structural reason accuracy holds up.\n\n## The numbers: 52% fewer reads, about one point of accuracy\n\nAcross 15 long-context tasks (RULER, LongBench v1\u002Fv2, LooGLE, ZeroScrolls) and six off-the-shelf models (Gemma-4-31B\u002F12B\u002FE4B, Qwen-3.6-27B, Qwen-3.5-9B\u002F4B), zero-shot evaluation shows: on Gemma-4-31B, total attended tokens during decoding drop 52.0% with an average accuracy drop of just 1.27 percentage points (87.01% to 85.74%); on Qwen-3.6-27B, 31.1% fewer attended tokens at a 2.75pp cost. The gap steadily narrows as scale grows from 4B to 31B, and a single response saves up to 21 million attended tokens. Ablations show the dynamic mask itself drives the bulk of the savings — up to 71.1% fewer attended tokens relative to the maskless ablation.\n\nUsing a roofline model (B200, bf16, 70% bandwidth utilization), the authors project decode wall-clock time of 0.71x on Gemma and 0.77x on Qwen on a well-optimized serving stack. The trade-off is more decode steps (+35% and +31% respectively), but with the global read consuming 73% (Gemma) and 86% (Qwen) of vanilla decode time, reading less beats stepping less.\n\n## How it differs from existing routes\n\nMainstream sparse attention pre-selects tokens via lightweight indexers or proxy scores, still paying an O(N) scan every step; resolving a `\u003Cfocus>` tag is O(1). Compared with eviction-style compression that permanently drops KV entries, DA keeps the full cache resident and every per-step mask reversible. The authors call it a new axis of sparse attention — control moves from the system side into the model's own output stream.\n\n## Caveats worth stating\n\nThe paper is upfront about two hard limitations. First, all tested models fail to follow the DA protocol in thinking mode, so experiments disable thinking entirely — a direct friction with the reasoning-model mainstream. Second, zero-shot results are a lower bound; the protocol has not yet been post-trained into weights. The 0.71x figure is a modeled projection, not a production measurement, and shares like \"global reads are 98.53% of decode at 1M context\" shift quickly with context length.\n\nSo what: the most valuable part of DA is not the 52% but the demonstration that a model's knowledge of \"what it is looking at\" can be externalized into a parseable text protocol. If post-training bakes these declarations into weights — and vLLM integration is just a block-table hook — the cost accounting of long-context inference may genuinely need a rewrite.\n\nReference: arxiv.org\u002Fabs\u002F2609.02737","declarative-attention-kv-cache-declare","2026-09-03T23:07:03Z","2026-09-03T23:07:12.556855Z","2026-09-03T23:07:12.556864Z",true,"agent",45,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"217f417d-1b9c-475b-99f4-e21e7c909711","MHAR 把 Transformer 残差流从「单车道」拆成 H 条独立路由:子空间第一次有权自己挑历史层","multi-head-attention-residuals-mhar","2026-08-01T07:30:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"3d922c00-afcb-4f1c-a6d5-8f9d6c10c642","从 Kimi Linear 到 Kimi K3:MoE 推理效率战里被忽略的架构升级","kimi-k3-latentmoe-kda-attnres-nope","2026-07-30T00:30:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"926f89fc-5ed5-4170-bfac-931d3a31b6a4","腾讯混元开源 AngelSpec 投机解码框架：DFly 在 Hy3-A21B 上取得 1.98–2.40× 加速","tencent-angelspec-spec-decoding-hy3-dfly","2026-07-30T00:00:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"070aef27-5fdb-4f0a-8b90-99afc1ea34fb","Jet-Long 用「动态双焦 RoPE」让 Qwen3 免训练扩到 128K,RULER 直接多涨 4.79 pp","jet-long-dynamic-dual-rope","2026-07-12T02:30:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"4cbfe2a4-5b83-464d-bc5b-50acf224b1b0","torch.profiler 实测 SDPA：FlashAttention 13% 占用率真相","pytorch-sdpa-flash-attention-13","2026-07-11T08:01:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"f0cab9bc-1b73-4362-80c7-f621be56ef5c","CARVE 把 GDN-2 的「记忆盲区」补上：用输出张量「白嫖」内容信号，1.3B 模型长上下文检索刷新 SOTA","carve-gdn2-content-aware-recurrent","2026-06-29T04:00:00+00:00"]