[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-diffusion-lm-fallback-commit-failure-roots":3,"topics-all":38,"news-related-76eb5d5a-04c1-4d7f-9c50-dc494b1c3bb8":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"76eb5d5a-04c1-4d7f-9c50-dc494b1c3bb8","扩散LM宽窗塌方解剖:一成回退提交,点燃七成失败","扩散语言模型靠并行写token提速,宽窗口却常让精度崩塌。一篇新论文逐个追踪四款开源DLM在数学、代码、翻译、摘要任务上的每一次提交,发现损失来自提交顺序而非并行度:回退提交只占总数11-35%,却引发61-72%的失败,早期EOS截断答案是最坏情况;只在最左未填位置提交,几乎不损失速度就能收回大部分精度。","扩散语言模型(DLM)一直有个尴尬的卖点:窗口越宽,一次 forward 能并行写出的 token 越多,理论上越快。可实际评测里,宽窗口经常让精度直接塌方——按这篇论文转述的 CD4LM 评测管线,LLaDA-8B 在 GSM8K 上从 32-token 块的 77.7% 崩到单个 256-token 块的 13.2%。11 月 16 日挂上 arXiv 的一篇论文(https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.20693,MBZUAI 与 UIUC 的三位研究者)把四个开源 DLM 的每一次 token 提交都翻出来查账,结论相当反直觉:塌方的锅主要不在「并行」,在「顺序」。\n\n## 先讲清楚机制\n\nLLaDA、Nemotron-Labs-Diffusion(NLD)、Fast-dLLM v2 这批开源 DLM 普遍用基于置信度的阈值解码:每一步,把窗口内所有置信度不低于 0.9 的未填位置一次性提交;如果一个都凑不出来,就由 fallback 把「最自信的那一个」硬提交上去。关键规则是提交永不撤回,而且可以乱序——左边还空着,右边就能先把词定了。论文把「最左边的未填位置」叫 frontier(前线),把左边未填时就抢先提交的行为叫 ahead commit。宽窗口给了 ahead commit 更大的活动空间,这就是嫌疑现场。\n\n## 自信度其实挤在前线附近\n\n论文的第一个发现:没必要抢跑。在四个模型上,15-24% 的位置要等自己成为 frontier 时才首次过阈值,60-64% 的位置在距 frontier 七个位置内过阈值,到距离 16 处首次过阈值的份额已跌破 1.5%。也就是说,能提前提交的 token 天生就少。实测里,frontier-only 策略(从 frontier 起把连续的自信位置一并提交)在全部 34 个设置下,每步提交 token 数仍有 vanilla 的至少 0.92 倍——NLD 上约一半、LLaDA 上约四分之三的 ahead commit 基本是白抢。更糟的是宽窗本身更贵:batch size 32 时,B=128 比 B=32 慢 2.4-2.5 倍。\n\n## 丢分的是顺序,不是并行度\n\n把「在哪提交」和「一次提交几个」拆开对比,天平一边倒。GSM8K 上,只在 frontier 提交的策略组比按置信度乱序提交的策略组高出 3 到 46 个点,而「每步提交一个还是几个」最多只动 2 个点。最夸张的是 LLaDA(评测提示 B):宽窗下 35.5% 对 frontier 系的 82.2%,差 46.4 个点;NLD-3B 用 256-token 窗口时,54.5% 对 88.0%,差 33.3 个点。精度损失几乎全由「提前把右边的词定了」造成。\n\n## 两种病因,每个模型的配方还不一样\n\n提前提交有两种死法,混配比例因模型而异。第一种是早期 EOS:end-of-sequence 被 fallback 从窗口深处提前拍死,答案直接被截断。NLD 上几乎全是这种病——把 EOS 限制在 frontier 提交后,顺序效应从 +8.4 点缩到 +1.1 点,B=256 时从 +33.3 缩到 +0.5。第二种是早期内容词:Fast-dLLM v2 上几乎全是这种,加了 EOS 限制顺序效应还是 +5.3 点纹丝不动。LLaDA 则两种病都有(+21.5 缩到 +17.8,+46.4 缩到 +24.8)。含义很实际:在某个模型上调好的修复,换一个模型可能直接失效。\n\n## 最硬的证据:回放实验\n\n论文用 root 分析加单步重放把因果钉死。失败解码与参考解码的第一个分叉点叫 root:fallback 提交只占总提交的 11-35%,却发起 61-72% 的有害分叉;逐 commit 做泊松回归后,在三个块扩散模型上,同等置信度下 ahead commit 引发分叉的频率仍是 frontier 提交的数倍(比率 7.8-14.6),而 fallback 提交开出有害 root 的频率是 confident 提交的 10.3-31.5 倍。重放实验更直接:把失败答案在 root 那一步改回只在 frontier 提交,比改动同样早的其他步骤多修复 7-39 个点的失败答案。元凶就是那一步。\n\n## 修复便宜得有点意外\n\n处方不用改权重、不用重训:要么只在 frontier 提交,要么至少把 fallback 和 EOS 关在 frontier。在 CD4LM 管线里,LLaDA 的 13.2% 加 EOS 限制回到 64.5%,每步都提交 frontier 则到 79.3%,而 forward pass 数量不变。距离帽 k≤2 也能保住精度,且论文扫过所有 k 值,没有一个能让速度提升超过 18%——抢跑在速度上几乎无利可图。论文代码已开源:https:\u002F\u002Fgithub.com\u002Frmakarovv\u002Ffast-at-the-frontier\n\n## 该泼的冷水\n\n四条边界要交代:模型都小于等于 8B;只研究了不撤回提交的阈值解码(DiffusionGemma 侧实验显示可撤回时 ahead 提交确实更常被改写,但主线没覆盖);任务是数学、代码、翻译、摘要这类从左到右自然的场景;精度效应主要出现在窗口宽于训练块时——不过 Fast-dLLM v2 从 8-token 子块扩到 32-token 训练块就已经掉了 4 个点,边界比想象中近。\n\n所以呢?并行解码的真正瓶颈从来不是窗口宽度,而是模型置信度的空间分布——训练要让模型在 frontier 前方更早自信,推理则老老实实把提交留在 frontier。宽窗口买到的不是速度,是「还没想好就先把话说出口」的机会。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.20693","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"6ad83302-b304-4ca4-b797-fd62648f380a","en","Fallback Commits Sink Diffusion LMs More Than Wide Windows","Four diffusion LMs traced commit by commit: fallback commits are 11-35% yet start 61-72% of failures; frontier-only decoding recovers most accuracy.","Diffusion language models (DLMs) are sold on one promise: a wider decoding window means more tokens written per forward pass, so decoding should get faster. In practice, wide windows often wreck accuracy. As reported in the paper, the CD4LM evaluation pipeline shows LLaDA-8B falling from 77.7% on GSM8K with 32-token blocks to 13.2% with a single 256-token block. A paper posted to arXiv on November 16 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.20693, by three researchers from MBZUAI and UIUC) traces every single token commit of four open-weight DLMs, and its verdict is counterintuitive: the collapse is caused mainly by commit order, not by parallelism.\n\n## How threshold decoding works\n\nLLaDA, Nemotron-Labs-Diffusion (NLD), and Fast-dLLM v2 all use confidence-based threshold decoding: at each step, every masked position in the window whose confidence passes 0.9 is committed at once; if none qualifies, a fallback commits the single most confident position. Commits are never revised, and they may happen in any order — a position can be filled while positions to its left are still blank. The paper calls the leftmost masked position the frontier, and a commit made while the left side is still masked an ahead commit. Wide windows give ahead commits room to roam, which is where the trouble starts.\n\n## Confidence huddles near the frontier\n\nThe first finding: rushing ahead is pointless. Across the four models, 15-24% of positions only pass the threshold once they themselves become the frontier, 60-64% pass it within seven positions of the frontier, and beyond distance 16 the share of positions passing for the first time drops below 1.5%. There simply are not many tokens worth committing early. In measurements, a frontier-only policy keeps at least 0.92 times the tokens-per-forward of vanilla across all 34 settings — roughly half of vanilla commits on NLD and about three quarters on LLaDA are ahead commits that add almost nothing. Wide windows are also expensive: at batch size 32, B=128 runs 2.4-2.5 times slower than B=32.\n\n## Order, not parallelism, loses the points\n\nSeparating where to commit from how many per step, the result is lopsided. On GSM8K, policies that commit only at the frontier beat confidence-order policies by 3 to 46 points, while committing one token per step versus several moves accuracy by at most 2 points. The widest gap: LLaDA with prompt B scores 35.5% under vanilla versus 82.2% for frontier policies, a 46.4-point difference; NLD-3B with a 256-token window sits at 54.5% versus 88.0%, a 33.3-point gap. The loss comes almost entirely from fixing words on the right before the text on the left exists.\n\n## Two failure modes, mixed differently per model\n\nAhead commits fail in two ways. One is early end-of-sequence: the fallback commits an EOS token from deep inside the window and the answer is cut short. This is nearly all of the damage on NLD — restricting EOS to the frontier shrinks the order effect from +8.4 points to +1.1, and from +33.3 to +0.5 at B=256. The other is early content tokens: on Fast-dLLM v2 nearly all of the loss remains (+5.3 points before and after the EOS restriction). LLaDA has both (+21.5 shrinks only to +17.8; +46.4 to +24.8). The practical lesson: a fix tuned on one model can simply fail on another.\n\n## The hardest evidence: replays\n\nThe paper nails causation with root analysis plus single-step replays. The first commit where a failing decode departs from the reference is the root: fallback commits are only 11-35% of all commits, yet they start 61-72% of the harmful departures. A per-commit Poisson regression shows that on the three block-diffusion models, an ahead commit still starts departures several times as often as a frontier commit at the same confidence (rate ratio 7.8-14.6), while a fallback commit starts harmful roots at 10.3 to 31.5 times the rate of a confident commit. In replays, repairing only the root step — switching it to frontier-only committing — fixes 7 to 39 points more failing answers than changing a matched earlier step.\n\n## The fix is embarrassingly cheap\n\nThe prescription changes no weights and requires no retraining: commit only at the frontier, or at least keep the fallback and EOS there. In the CD4LM pipeline, LLaDA climbs from 13.2% to 64.5% with the EOS restriction, and to 79.3% committing the frontier at every step, with the same number of forward passes. A distance cap of k at most 2 keeps frontier-only accuracy, and no cap value increases speed by more than 18% — there is almost nothing to gain from committing ahead. The code is released at https:\u002F\u002Fgithub.com\u002Frmakarovv\u002Ffast-at-the-frontier\n\n## Caveats worth stating\n\nFour boundaries: the models are at most 8B parameters; the study covers threshold decoding that never revises a commit (a side experiment on DiffusionGemma, where revision is allowed, finds ahead commits are indeed revised more often, but the main line does not cover it); the tasks — math, code, translation (WMT16), summarization (CNN\u002FDailyMail) — are ones where left-to-right order is natural; and the big accuracy effects appear when the window is wider than the training block, though Fast-dLLM v2 already loses 4 points going from its default 8-token sub-block to its 32-token training block.\n\nSo what? The real bottleneck of parallel decoding was never window width but the spatial spread of the model's confidence — training should make the model confident further ahead of the frontier, and inference should keep commits at the frontier. A wide window does not buy speed; it buys the chance to speak before you have thought.","diffusion-lm-fallback-commit-failure-roots","2026-11-17T15:35:00Z","2026-11-17T15:32:14.895534Z","2026-11-17T15:32:14.895542Z",true,"agent",29,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"269851b4-d6e6-43a6-9239-b3116708b735","块边界是同步税:扩散LM重叠去噪免训练提速14.5%","b3-overlapping-block-denoising-dlm","2026-11-11T17:11:59+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"51953ff7-5702-4d21-9af6-92d8edf7af09","块扩散LM双稀疏提速:不动权重,32K单步快4.72倍","duet-dual-sparse-bdlm-inference","2026-11-09T15:20:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"fd9298b5-dea9-4641-8f73-56ae32261c3e","扩散LM投机采样:串行评估砍到1\u002F4,分布零走样","specctmc-exact-speculative-diffusion-lm","2026-11-07T15:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"af6bfcff-1ffa-400d-ac93-cb6803ab232c","dSpark调度蒸馏:扩散LLM解码最多快10倍","dspark-schedule-distillation-diffusion-llm","2026-10-27T17:09:34+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"a781accb-537d-4a09-ad5a-d26b87612801","扩散LLM成本实测:写故事比改语法贵3倍,错答案最烧算力","diffusion-llm-decoding-cost","2026-10-25T21:09:54+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"6ede1566-74c5-4bdd-9dd5-6d3f118db876","投机解码的重复陷阱:D-Loop 两轮循环破局","d-loop-diffusion-speculative-decoding","2026-10-06T15:20:00+00:00"]