[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-d3llm-icml-2026-pseudo-trajectory-5x":3,"topics-all":37,"news-related-22588d44-ca1a-4cbd-9e39-1dfb336f0ec5":56},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":35,"view_count":36},"22588d44-ca1a-4cbd-9e39-1dfb336f0ec5","d3LLM：ICML 2026 提出伪轨迹蒸馏，让扩散语言模型提速 5 倍","主流大语言模型几乎全部采用自回归（AR）生成范式，token 一个接一个顺序输出。这种方式在推理速度上存在天然瓶颈——序列越长，生成耗时越多。近一年来，扩散语言模型（dLLM）作为颠覆性替代方案受到关注，Google Gemini Diffusion、Inception Mercury 等先后发布，承诺并行解码带来的更高吞吐量。\n\n但速度与精度真的可以兼得吗？ICML 2026 接收的论文 d3LLM（pseuDo-Distilled Diffusion LLM）直面这个问题，提出了一套系统性的解决思路。\n\n**核心创新：伪轨迹蒸馏**\n\nd3LLM 由 Hao-AI Lab 团队提出，发表在 arXiv（2601.07568）。研究团队发现，现有扩散模型在随机 masking 策略下学习效率偏低——学生模型不知道教师模型喜欢先解码哪些","https:\u002F\u002Farxiv.org\u002Fabs\u002F2601.07568","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"7b878b02-7b01-4b6c-afe1-9b5c8b977b78","en","d3LLM: pseudo-trajectory distillation, 5x faster diffusion LMs","Mainstream large language models almost universally use the autoregressive (AR) generation paradigm, outputting tokens one after another. This approach has a natural inference-speed bottleneck — the longer the sequence, the more generation time. In the past year, diffusion language models (dLLM) have attracted attention as a disruptive alternative; Google Gemini Diffusion, Inception Mercury, and others have launched, promising the higher throughput of parallel decoding.\n\nBut can speed and accuracy really coexist? The ICML 2026 paper d3LLM (pseuDo-Distilled Diffusion LLM) tackles this question head-on, proposing a systematic solution.\n\n**Core innovation: Pseudo-Trajectory Distillation**\n\nd3LLM, from the Hao-AI Lab team, was published on arXiv (2601.07568). The team found that existing diffusion models learn inefficiently under random masking strategies — the student model doesn't know which tokens the teacher model prefers to decode first. So they extracted Pseudo-Trajectories from the teacher model's real decoding order, using them to guide the student model toward a generation order closer to that of autoregressive models; this design alone brought an 18% TPF (Token Per Forward) improvement.\n\nAdditionally, the team designed a Progressive Noise Schedule and a Progressive Window Sizing strategy, contributing an extra 12% and 8% TPF improvement respectively.\n\n**Inference side: multi-block parallel decoding**\n\nAt decoding, d3LLM introduces entropy-based multi-block parallel decoding — each forward pass handles multiple blocks simultaneously, with entropy values used to filter high-confidence tokens. Combined with periodic KV-Cache refresh, throughput in long-context scenarios is boosted another 35%. Layered with the EOS early-stop mechanism, the overall system achieves up to 5× speedup (vs. Qwen-2.5-7B-it on H100 GPU).\n\n**Accuracy-speed trade-off: introducing the AUP metric**\n\nOne of the paper's key contributions is pointing out the trade-off law between accuracy and parallelism: processing more tokens per forward pass almost inevitably comes with accuracy loss. To address this, the team proposes the AUP (Accuracy Under Parallelism) metric, which holistically measures the speed-accuracy balance, helping the community more fairly evaluate the cost-performance of diffusion language models versus autoregressive models.\n\n**Why this is worth attention**\n\nd3LLM's significance is not in dethroning autoregression, but in proving that the diffusion paradigm can, in engineering terms, very closely match or even exceed the autoregressive baseline. With 5× speedup plus native support from mainstream inference engines like SGLang, dLLM's path from lab to production is now clear. For cost-sensitive deployment scenarios, this is a direction worth continued attention.","d3llm-icml-2026-pseudo-trajectory-5x","2026-05-21T16:01:00Z","2026-05-21T16:04:03.214376Z","2026-08-19T02:08:40.142862Z",true,"agent","token。于是他们从教师模型的真实解码顺序中提取伪轨迹（Pseudo-Trajectory），以此引导学生模型学习更接近自回归模型的生成顺序，这一设计单独带来了 18% 的 TPF（Token Per Forward）提升。\n\n此外，团队还设计了渐进式噪声调度（Progressive Noise Schedule）和渐进式窗口大小策略（Progressive Window Sizing），分别带来 12% 和 8% 的额外 TPF 提升。\n\n**推理侧：多 block 并行解码**\n\n在解码阶段，d3LLM 引入基于熵的多 block 并行解码——每次 forward pass 同时处理多个 block，用熵值筛选高置信度 token。同时配合周期性 KV-Cache 刷新，使长上下文场景下的吞吐量再提升 35%。结合 EOS 早停机制，整体实现最高 5 倍加速（对比 Qwen-2.5-7B-it，在 H100 GPU 上）。\n\n**精度-速度权衡：引入 AUP 指标**\n\n论文的关键贡献之一是指出了精度-并行之间的权衡规律：每次 forward pass 处理更多 token，几乎必然伴随精度损失。为此团队提出了 AUP（Accuracy Under Parallelism）指标，综合衡量速度与精度的平衡，帮助社区更公平地评估扩散语言模型与自回归模型的性价比。\n\n**为什么这值得关注**\n\nd3LLM 的意义不在于打倒自回归，而在于证明了扩散范式在工程上可以非常接近甚至超越自回归基线。当 5 倍加速配合 SGLang 等主流推理引擎的原生支持，dLLM 从实验室走向生产环境的路径已经清晰。对推理成本敏感的部署场景，这是一个值得持续关注的方向。",202,[38,47],{"slug":39,"tag_slug":39,"title_zh":40,"title_en":41,"intro_zh":42,"intro_en":43,"id":44,"is_active":33,"created_at":45,"modified_at":46},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":48,"tag_slug":48,"title_zh":49,"title_en":50,"intro_zh":51,"intro_en":52,"id":53,"is_active":33,"created_at":54,"modified_at":55},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":57},[58,63,68,73,78,83],{"id":59,"title":60,"news_slug":61,"published_at":62},"8914cfad-53bc-4f76-bb06-8579d23cfa5d","AR 大模型外挂 diffusion 权重:Uno 免草稿模型实现 3 倍无损解码加速","uno-diffusion-augmented-llm-speedup","2026-09-08T13:30:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"bd1a9589-0cca-4f68-a90d-3e454e94554f","Bifocal dLLM：Mamba 旁路解 KV 困局，吞吐 2.4×–12.9×","bifocal-dllm-r2lm-mamba-qwen3-1-7b","2026-06-29T10:08:00+00:00",{"id":69,"title":70,"news_slug":71,"published_at":72},"3dab673e-0bdc-442a-9670-87964ebf8f79","Dynamic-dLLM：动态缓存预算+自适应并行解码，给扩散语言模型提速 3 倍","dynamic-dllm-cache-budget-3x","2026-06-25T10:00:00+00:00",{"id":74,"title":75,"news_slug":76,"published_at":77},"0b744f65-a3d5-47d8-84d1-eb25a7a2798e","FMLM+ 把扩散语言模型的「自纠错」解锁：32× 更少 NFE 匹配离散基线","fmlm-plus-posterior-refinement-32x-nfe","2026-06-25T02:30:00+00:00",{"id":79,"title":80,"news_slug":81,"published_at":82},"80bc5e25-d24d-45a1-9c2b-534ebcae39f9","腾讯 WeDLM 开源：让扩散 LLM 在标准因果注意力下跑出 3-6× vLLM 加速","tencent-wedlm-diffusion-llm-causal-3-6x","2026-06-16T20:00:00+00:00",{"id":84,"title":85,"news_slug":86,"published_at":87},"634ba8f7-771e-4492-aacb-549112a8c91a","EPIC 让扩散语言模型重获并行优势：CFG 约束解码推理时间压缩 67.5%","epic-cfg-dllm-67-5pct-speedup","2026-06-09T16:00:00+00:00"]