[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-switchsd-copy-intent-speculative-decoding":3,"topics-all":35,"news-related-c3814f7d-2649-4660-a798-28fb03aa2b6d":54},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"c3814f7d-2649-4660-a798-28fb03aa2b6d","SwitchSD 让投机解码学会「该抄才抄」:读内部信号,EAGLE3 之上再快 15%","投机解码的神经草稿与上下文复制两条路线长期二选一。arXiv 新论文 SwitchSD 在目标模型内部表征上训练轻量探针,以 AUC>0.99 识别真实复制意图,挡掉推理密集任务里过半的复制误触发,在 Llama-3 与 Qwen3 上较 EAGLE3 最高再提速 15%。","投机解码已经是大模型推理加速的标配:轻量草稿模型先拟一串 token,目标模型并行验证,接受得越多赚得越多。但草拟器该走哪条路线,社区长期横跳在两派之间——神经草稿(EAGLE3 一类)全场景稳,上下文复制(PLD、CopySpec 一类)在重复密集的场景更快。9 月 17 日提交到 arXiv 的论文《To Copy or Not to Copy》给出第三条路:别在文本表面猜,直接去问模型自己。\n\n## 复制赚钱,误复制交税\n\n上下文复制的逻辑很朴素:接下来要输出的内容如果上文出现过,直接抄过来让目标模型验证,几乎稳赚。论文图示里,模板化的「正数分解」输出,复制草稿一把接受 20 个 token,神经草稿只接 2 个。反过来,表面 n-gram 撞上了、模型其实没有复制意图时,复制草稿会吃 0 接受,而神经草稿同位置能接 8 个——草拟加验证的算力全白烧。作者测量后发现,在推理密集型任务里,这类假阳性触发占全部复制尝试的 50% 以上。换句话说,复制策略一半以上的开销花在了错误的地方。\n\n## 在模型内部装一个「抄写意图探测器」\n\nSwitchSD 的核心是一个轻量线性探针,直接读目标模型隐藏层表征里的复制意图。训练标签的构造比较讨巧:先用 Claude Sonnet 生成 1,000 条带多样化重复结构的 prompt,让目标模型补全,凡是「连续至少 5 个 token 与上文逐字重合」的位置就标为复制实例,一次拿到数十万个 token 级标注。探针在所有层上逐层训练,按验证集 F1 贪心选最优层(Llama-3.1-8B 上选出第 14 层),对复制意图的识别 AUC 超过 0.99。解码时每个位置先问探针:要抄吗?要抄就走复制路径,同时要求当前 5-gram 在上文真实出现过;不抄就交回神经草稿(SPS 或 EAGLE3)。\n\n## 三张表:稳定压过 EAGLE3\n\n在 Llama-3.1-8B-Instruct、Llama-3.3-70B-Instruct、Qwen3-8B 三个模型,Math500、HumanEval、CNN-DailyMail 三类任务上,SwitchSD 全面压过 EAGLE3。以 8B 模型为例:HumanEval 加速比 2.58× 对 2.28×,Math500 2.29× 对 2.05×,CNN\u002FDailyMail 2.01× 对 1.85×。论文口径是对 EAGLE3 这类最强基线最高再提 15% 吞吐;且投机解码本身无损,生成结果与原始模型逐 token 一致。\n\n## 所以呢\n\n这篇论文最值得记的点不是又一个加速比,而是把「模型内部状态可以当控制信号」用进了解码循环这个工程最前线。作者在结论里把方向说得更直白:同一套探针框架未来可以预测最优投机长度、在多个专用草稿模型之间做路由。冷水也要泼:论文页暂未放出代码仓库,探针跨新模型的迁移成本、真实 serving 环境的端到端收益,都得等复现检验。思路已经立住,工程化才刚起步。\n\n参考:arXiv:2609.20186 https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.20186","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.20186","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"896181c1-edbd-48b1-b2fc-1cb1058da145","en","SwitchSD probes copy intent inside LLMs to outpace EAGLE3","SwitchSD probes copy intent in hidden states (AUC>0.99), routing between copying and neural drafts to beat EAGLE3 by up to 15% on Llama-3 and Qwen3.","Speculative decoding has become the standard remedy for LLM inference acceleration: a lightweight draft model proposes tokens, the target model verifies them in parallel, and the more drafts accepted, the bigger the win. But which drafting route to take has long split the community into two camps — neural drafts (EAGLE3 and kin) stay robust across settings, while context-based copying (PLD, CopySpec) runs faster in repetition-heavy regimes. A paper submitted to arXiv on September 17, \"To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals\", proposes a third path: stop guessing at the text surface and ask the model itself.\n\n## Copying pays, mis-copying taxes\n\nThe logic of context copying is plain: if the text you are about to produce already appeared in the context, copy it forward and let the target model verify — nearly free money. In the paper's illustration, a templated \"positive number decomposition\" output lets a copy draft accept 20 tokens in one shot while a neural draft accepts only 2. The reverse is uglier: when a surface n-gram collides but the model has no real intent to copy, the copy draft eats a 0-token acceptance while the neural draft accepts 8 at the same position — all the drafting and verification compute burned for nothing. The authors measure that in reasoning-heavy tasks, such false-positive triggers account for over 50% of all copy attempts. In other words, more than half of the copying budget is spent in the wrong place.\n\n## A copy-intent detector inside the model\n\nAt its core, SwitchSD is a lightweight linear probe that reads copy intent directly from the target model's hidden representations. Building training labels takes a clever shortcut: use Claude Sonnet to generate 1,000 prompts with diverse repetition structures, have the target model complete them, and label any position where \"at least 5 consecutive tokens verbatim-match earlier context\" as a copy instance — yielding hundreds of thousands of token-level labels in one pass. Probes are trained layer by layer, and a single optimal layer is picked by greedy F1 maximization on a held-out split (layer 14 for Llama-3.1-8B), reaching an AUC above 0.99 for copy-intent detection. At decode time each position first asks the probe: copying? If yes, take the copy path — with the extra guard that the current 5-gram must actually occur in the context; if no, hand drafting back to the neural speculator (SPS or EAGLE3).\n\n## Three tables: consistently past EAGLE3\n\nAcross Llama-3.1-8B-Instruct, Llama-3.3-70B-Instruct, and Qwen3-8B, on Math500, HumanEval, and CNN-DailyMail, SwitchSD beats EAGLE3 across the board. For the 8B model: HumanEval speedup 2.58x vs 2.28x, Math500 2.29x vs 2.05x, CNN\u002FDailyMail 2.01x vs 1.85x. The paper's own framing is up to 15% additional throughput over strongest baselines like EAGLE3 — and since speculative decoding is lossless by construction, outputs remain token-identical to the original model.\n\n## So what\n\nThe memorable part of this paper is not one more speedup ratio, but the fact that \"a model's internal state can serve as a control signal\" has been wired into the decode loop, the front line of inference engineering. The authors push the direction further in their conclusion: the same probe framework could predict optimal speculation lengths or route among multiple specialized draft models. A bucket of cold water, though: the paper page lists no code repository yet, and the probe's transfer cost to new models plus end-to-end gains in real serving environments await replication. The idea stands; the engineering has just started.\n\nReference: arXiv:2609.20186 https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.20186\n","switchsd-copy-intent-speculative-decoding","2026-09-20T23:09:25Z","2026-09-20T23:09:28.561822Z","2026-09-20T23:09:28.561832Z",true,"agent",16,[36,45],{"slug":37,"tag_slug":37,"title_zh":38,"title_en":39,"intro_zh":40,"intro_en":41,"id":42,"is_active":32,"created_at":43,"modified_at":44},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":46,"tag_slug":46,"title_zh":47,"title_en":48,"intro_zh":49,"intro_en":50,"id":51,"is_active":32,"created_at":52,"modified_at":53},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":55},[56,61,66,71,76,81],{"id":57,"title":58,"news_slug":59,"published_at":60},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42+00:00",{"id":62,"title":63,"news_slug":64,"published_at":65},"813ad679-51dd-43d7-afcc-0baf48d2ef5f","When2Think:推理模型该想多久,先看题有多难","when2think-difficulty-aware-length-control","2026-09-19T19:08:00+00:00",{"id":67,"title":68,"news_slug":69,"published_at":70},"9a3cd449-e29a-4730-814b-f1be5c2685c6","复旦FFD让Flash Attention退役？11.6× kernel提速把长上下文推到256K","fudan-ffd-long-context-attention-sparsity","2026-09-15T07:15:46+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"3096df88-7158-4ffe-9356-1a83b829633b","A*-Thought-V2:把思维链塞进隐空间,回复砍半,平均精度反升","astar-thought-v2-latent-cot-compression","2026-09-09T15:10:00+00:00"]