[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-prefix-sliding-efficient-test-time-scaling":3,"topics-all":38,"news-related-5c75bd80-9f12-499a-898f-019615ac98ee":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"5c75bd80-9f12-499a-898f-019615ac98ee","Prefix Sliding:让推理模型长思考提速3倍的免训练方案","arXiv 新论文 Prefix Sliding 提出:推理模型长思考时只对指令前缀和最近几千 token 保持注意力,丢弃失活的中间推理 token,把内存需求封顶。免训练下现有模型提速 3 倍不掉分,RL 训练可扩展到 10 万+ token 的推理轨迹。","推理模型的\"思考\"是要付显存账单的。当模型用 test-time scaling 解难题时,它会生成很长的推理链,而主流架构通过 full attention 把整条推理轨迹都留在内存里——想得越久,KV 缓存越大,算力和显存成本跟着膨胀。难题恰恰需要长思考,于是最需要算力的场景就是最贵的场景。\n\n## 核心洞察:中间 token 会\"失活\"\n\n8 月 26 日上线 arXiv 的论文《Prefix Sliding for efficient test-time scaling》(编号 2608.26070)给出了一个反直觉的观察:随着模型继续推理,大多数中间推理 token 会逐渐失去重要性。既然这些 token 后面基本用不到了,一直留着它们还值得付出成本吗?这个洞察直接质疑了一个隐含假设——推理轨迹里每个 token 都同等重要、都必须全程参与注意力计算。\n\n## 方法:前缀 + 滑动窗口\n\n论文提出 Prefix Sliding:推理过程中只保留两类 token 的注意力——prefix(任务指令、可用工具等关键信息)和最近几千 token 组成的滑动窗口(模型当前正在进行的推理),其余中间 token 直接丢弃。这样无论模型推理多长,总内存需求都被封顶。\n\n免训练直接用在现有模型上,论文报告可提速 3 倍且保持性能;配合强化学习训练,还能把推理轨迹扩展到十万 token 以上——这是 full attention 内存预算下很难触达的区间。消融实验显示,这个方案优于\"把中间 token 摘要压缩\"和\"朴素滑动窗口\"两个直觉替代品。\n\n## 工程侧:落在 vLLM 和 flash-attn 上\n\n方法不是纸上谈兵。GitHub 仓库(Muennighoff\u002Fprefix-sliding,Apache-2.0 许可证)给出了基于 vLLM 和 flash-attn 分支的实现,README 演示了在 Qwen3-1.7B 上开启 4096 滑窗做 32K 长度生成;评测覆盖 AIME、GPQA、MATH500、HealthBench 和 LiveCodeBench(后者跑到 262144 max tokens)。评测结果文件和 RL 训练数据集(prefixsliding\u002Ftrain_v6_filtered)都放在 HuggingFace 上,RL 部分对接 prime-rl 与 trl 两套框架。\n\n作者阵容也值得注意:一作 Niklas Muennighoff 之外,还有 Percy Liang、Jason Wei、Andrew Y. Ng、Yejin Choi、Luke Zettlemoyer、Mike Lewis 等,论文署名共 18 人,全文 28 页(正文 9 页)。这种配置通常意味着方法会被社区快速跟进复现。\n\n## 所以呢\n\ntest-time scaling 的主流叙事是\"多想 = 多分\",但成本曲线决定它能不能普及。Prefix Sliding 的意义在于把\"想得久\"的边际显存成本压成常数——这对推理 API 定价、端侧长推理、agent 长任务都是直接利好。更值得记住的是那个洞察本身:推理链里的中间步骤是易腐品,不是资产。下次看到模型\"思考了十万 token\",不妨想一想:其中有多少真的需要被记住?\n\n论文:https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.26070\n代码:https:\u002F\u002Fgithub.com\u002FMuennighoff\u002Fprefix-sliding","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.26070","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"045c011e-e2bb-45ce-bdd6-0c927f8a3b87","token-efficiency",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"f0ccbd19-954d-472e-b113-f37d9035d25c","en","Prefix Sliding: 3x Faster Long Reasoning Without Retraining","A new arXiv paper, Prefix Sliding, proposes that reasoning models keep attention only on the instruction prefix and the last few thousand tokens, discarding stale intermediate reasoning tokens to cap memory. Without training, existing models run 3x faster with performance maintained; RL training scales reasoning beyond 100K tokens.","Reasoning models pay a memory bill for their \"thinking.\" When a model uses test-time scaling to solve hard problems, it generates a long reasoning chain, and mainstream architectures keep that entire trace in memory via full attention — the longer it thinks, the larger the KV cache grows, and compute and memory costs balloon with it. Hard problems are exactly the ones that need long thinking, so the scenarios that need the most capability are also the most expensive ones.\n\n## The core insight: intermediate tokens go \"stale\"\n\nA paper published on arXiv on August 26, \"Prefix Sliding for efficient test-time scaling\" (2608.26070), starts from a counterintuitive observation: as the model continues reasoning, most intermediate reasoning tokens lose importance. Since those tokens are barely needed later, is retaining them worth the cost? This insight directly challenges a hidden assumption — that every token in a reasoning trace matters equally and must participate in attention for the whole run.\n\n## The method: prefix + sliding window\n\nThe paper proposes Prefix Sliding: during reasoning, attention covers only two groups of tokens — the prefix (key instructions, available tools, and other metadata) and a sliding window of the last few thousand tokens (the reasoning the model is currently working on) — while intermediate tokens in between are simply discarded. This caps total memory requirements no matter how long the model reasons.\n\nApplied to existing models without any training, the paper reports a 3x speedup while maintaining performance; combined with reinforcement learning training, it enables scaling to reasoning traces beyond a hundred thousand tokens — a regime that is hard to reach under full-attention memory budgets. Ablations show the approach outperforms two intuitive alternatives: summarizing intermediate tokens, and a vanilla sliding window.\n\n## The engineering: it lands on vLLM and flash-attn\n\nThis is not a paper-only proposal. The GitHub repository (Muennighoff\u002Fprefix-sliding, Apache-2.0 license) provides an implementation built on vLLM and flash-attn branches; the README demonstrates enabling a 4096-token sliding window on Qwen3-1.7B for 32K-length generation. Evaluation covers AIME, GPQA, MATH500, HealthBench, and LiveCodeBench (the latter run at 262144 max tokens). Evaluation result files and the RL training dataset (prefixsliding\u002Ftrain_v6_filtered) are hosted on HuggingFace, and the RL side hooks into both prime-rl and trl.\n\nThe author lineup is also notable: beyond first author Niklas Muennighoff, the paper lists Percy Liang, Jason Wei, Andrew Y. Ng, Yejin Choi, Luke Zettlemoyer, and Mike Lewis among its 18 authors, spanning 28 pages (9 main). That configuration usually means the community will replicate and build on the method quickly.\n\n## So what\n\nThe mainstream narrative of test-time scaling is \"think longer, score higher,\" but the cost curve decides whether it can spread. The significance of Prefix Sliding is that it turns the marginal memory cost of thinking long into a constant — a direct win for reasoning API pricing, on-device long reasoning, and long-horizon agents. The more durable takeaway is the insight itself: intermediate steps in a reasoning chain are perishable goods, not assets. Next time you see a model \"think for a hundred thousand tokens,\" ask yourself: how much of that really needed to be remembered?\n\nPaper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.26070\nCode: https:\u002F\u002Fgithub.com\u002FMuennighoff\u002Fprefix-sliding","prefix-sliding-efficient-test-time-scaling","2026-08-27T17:20:00Z","2026-08-27T17:11:05.811434Z","2026-08-27T17:11:05.811445Z",true,"agent",170,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"3096df88-7158-4ffe-9356-1a83b829633b","A*-Thought-V2:把思维链塞进隐空间,回复砍半,平均精度反升","astar-thought-v2-latent-cot-compression","2026-09-09T15:10:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"8a42c9c3-a1c7-40fb-8c75-8ac42977b5af","D-cut 把投机解码的「长草稿」剪掉一半：高并发推理平均提速 1.65×、MoE 跑出 3×","d-cut-speculative-draft-cut","2026-07-18T10:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"21be10ba-3364-4b55-a248-e6f878dae68b","QuasiMoTTo：quasi-Monte Carlo 进 test-time scaling","quasimotto-quasi-monte-carlo","2026-07-02T10:20:52+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"a453eb28-7fb0-4e07-adc1-0d0575850758","EntMTP 用熵信号给多 token 推测装上调速器：让 LLM 自适应匹配上下文可预测性","entmtp-entropy-speculative-decoding","2026-06-29T12:21:51+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9a3cd449-e29a-4730-814b-f1be5c2685c6","复旦FFD让Flash Attention退役？11.6× kernel提速把长上下文推到256K","fudan-ffd-long-context-attention-sparsity","2026-09-15T07:15:46+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00"]