[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-specopd-branch-parallel-speculative-distillation":3,"topics-all":38,"news-related-19637bd9-df50-4ecb-87e5-2a4ab633ddff":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"19637bd9-df50-4ecb-87e5-2a4ab633ddff","SpecOPD并行采分支:投机解码训练吞吐16倍","SpecOPD 把投机解码 drafter 的在线蒸馏从串行 rollout 改为参考锚点并行采分支,提案树共享前缀计算,兼容 GQA\u002FQSA\u002FGDN 三种注意力。论文报告接受长度最高提升 41.45%、监督 token 吞吐 16.09 倍、生成加速最高 3.56 倍,覆盖 Qwen 三档模型与四类任务。","投机解码(speculative decoding)是目前大模型推理提速的主流路线之一:小 drafter 先猜一批 token,目标模型一次性并行验证,猜中的直接采纳。但决定整体加速的不只是 drafter 猜得多快,更关键的是接受长度(acceptance length)——每轮验证能留下几个 token。监督微调在固定响应上训练 drafter,容易撞上接受长度平台期;在线蒸馏(on-policy distillation,OPD)把训练挪到 drafter 自己采样的块上、保留逐 token 的目标监督,被证明能推过平台期。麻烦在于,传统 OPD 管线要靠「drafter 提案、目标验证、接受前缀决定下一轮起点」的串行循环收集监督信号,rollout 与训练交替执行,论文实测收集阶段 GPU 利用率明显偏低。\n\narXiv 11 月 2 日提交的 SpecOPD([arXiv:2611.02846](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.02846),15 位作者,Hugging Face 论文页挂着 Qwen 组织标识)问了一个更根本的问题:在线监督凭什么必须继承这套串行依赖?他们的结论是——不必。监督一个 draft block 只需要参考前缀加上它自己采样的历史,不需要等一条完整响应被逐轮验证完。\n\n## 把 rollout 拆成一棵提案树\n\nSpecOPD 把参考序列当共享主干,在主干上随机取一组锚点,让当前 drafter 在每个锚点后并行采样独立分支,再把分支打包成提案树:主干只 prefill 一次,全部分支共享前缀计算,由目标模型并行打分。工程难点在分支历史不能互相污染,论文按注意力类型给出三套方案:GQA 全注意力用树形掩码;QSA 稀疏注意力用分支一致的 indexer 组;GDN 线性注意力的递归状态没法靠掩码隔离,改用索引状态回放。训练目标上,E2E-TV 替代损失直接把优化对准接受长度本身。\n\n## 三个数字,一张评测矩阵\n\n论文报告:平均接受长度最高提升 41.45%;相对串行 rollout OPD,监督 token 吞吐 16.09 倍;相对自回归解码,生成加速最高 3.56 倍。评测矩阵铺得够开:目标模型覆盖 Qwen3-8B(GQA)、Qwen3.8-27B(GQA-GDN 全线性)、Qwen3.8-Flash-Next-125B(QSA-GDN 稀疏线性)三档;drafter 有 DFlash、DSpark、DFlash V2 三种并行架构;任务覆盖对话、编码、数学、agentic 四域。\n\n## 冷静看的部分\n\n两点先按住。其一,16.09 倍是训练侧「监督 token 吞吐」口径,不是端到端训练时间,更不是推理加速;3.56 倍也是「最高」值而非平均。其二,作者在 Hugging Face 社区帖说明:首发仓库只含项目概览与图表,训练代码、评测脚本、模型权重都不在首发版本里,Hugging Face 上也暂无模型关联这篇论文——论文先行、工件在路上,复现要等后续。\n\n对做推理服务的人,这篇的价值不在数字,而在视角:当并行 drafter 架构收敛成少数几条主流路线,瓶颈已经从「设计更好的 drafter」挪到「更快地训练 drafter」,训练吞吐本身成了系统能力。下次看到厂商宣传推理提速,值得先问一句:他们的 drafter,多久重训一次?\n\n参考:[arXiv:2611.02846](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.02846) · [Hugging Face 论文页](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2611.02846)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.02846","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"1eb277ee-c8c2-4455-b28d-df7819069822","en","SpecOPD: parallel branches lift draft training throughput 16.09x","SpecOPD supervises speculative drafters in parallel branches: acceptance length up 41.45%, training throughput 16.09x, decoding 3.56x faster.","Speculative decoding speeds up large language model inference by letting a lightweight drafter propose tokens that the target model verifies in parallel. But the overall speedup does not only depend on how cheaply the drafter proposes; it hinges on acceptance length — how many tokens survive verification in each round. Supervised fine-tuning on fixed responses tends to plateau there, and on-policy distillation (OPD), which trains the drafter on its own sampled blocks under dense token-level target supervision, has been shown to push past that plateau. The catch: conventional OPD pipelines collect supervision through sequential draft-and-verify rollouts, alternating between rollout collection and training, with low GPU utilization during collection.\n\nSpecOPD, submitted to arXiv on November 2 ([arXiv:2611.02846](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.02846), 15 authors, carrying the Qwen organization badge on its Hugging Face paper page), asks a more basic question: why must on-policy supervision inherit those sequential dependencies? The authors' answer: it does not have to. A draft block can be supervised using only its reference prefix and its own draft-sampled history — no complete verified response required.\n\n## Turning rollouts into a proposal tree\n\nSpecOPD treats the reference sequence as a shared trunk, samples a set of anchors along it, and lets the current drafter sample independent branches in parallel behind each anchor. The branches are packed into a proposal tree: the trunk is prefilled once, all branches share the prefix computation, and the target scores them in parallel. The hard engineering problem is keeping each branch's history isolated. The paper gives one mechanism per attention type: tree attention masks for full attention (GQA), branch-consistent indexer groups for sparse attention (QSA), and indexed state replay for linear attention (GDN), whose recurrent states cannot be isolated by masking alone. Training uses an E2E-TV surrogate loss that directly targets acceptance length.\n\n## Three numbers, one evaluation matrix\n\nThe paper reports up to 41.45% higher mean acceptance length, 16.09x supervised-token throughput over sequential-rollout OPD, and up to 3.56x generation speedup over autoregressive decoding. The evaluation matrix is broad: targets span Qwen3-8B (GQA), Qwen3.8-27B (GQA-GDN, full-linear) and Qwen3.8-Flash-Next-125B (QSA-GDN, sparse-linear); drafters cover DFlash, DSpark and DFlash V2; tasks cover dialogue, coding, mathematics and agentic workloads.\n\n## The sober part\n\nTwo caveats. First, 16.09x is a training-side supervised-token throughput figure, not end-to-end training time and not inference speedup; 3.56x is a best-case number. Second, the authors state on the Hugging Face paper page that the initial repository contains only the project overview and figures — training code, evaluation scripts and model weights are not included, and no models currently link to the paper. Paper first, artifacts later; reproduction will have to wait.\n\nFor inference teams, the value here is the shift in perspective: as parallel drafter architectures converge to a few mainstream routes, the bottleneck moves from designing a better drafter to training the drafter faster — training throughput itself becomes a systems capability. Next time a vendor advertises an inference speedup, ask one question first: how often do they retrain their drafter?\n\nReferences: [arXiv:2611.02846](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.02846) · [Hugging Face paper page](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2611.02846)","specopd-branch-parallel-speculative-distillation","2026-11-05T23:20:00Z","2026-11-05T23:07:22.910219Z","2026-11-05T23:07:22.910231Z",true,"agent",50,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"931283f8-6928-4128-bc6c-2d11ba240fca","397B混合LLM免重训:长文本解码提速2.2倍","hybrid-llm-training-free-sparse-decoding","2026-11-05T13:08:09+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"77a567e5-2209-4981-882b-e2a3077a0012","先猜再听:全双工语音模型把投机解码搬到音频到来之前","speculative-listening-full-duplex-speech-decoding","2026-11-06T09:05:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"53d00d00-e4fb-41b0-a36a-37161203b514","KV缓存拼接复用丢题可至100%,修补要盯答案块","kv-cache-repair-budget-target-chunk","2026-11-05T15:09:02+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"d456840a-1a1d-412d-90ac-73ca5a5bbe10","MoE 路由器读出推理状态:砍规划省三成 token","moe-router-reasoning-behavior-steering","2026-11-05T09:21:56+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"64088ca6-c244-474c-93ce-cdbfe55de6d6","投机解码没用满?agent循环把加速比挖到10.87倍","specpilot-agentic-speculative-decoding","2026-11-04T13:15:17+00:00"]