[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-specctmc-exact-speculative-diffusion-lm":3,"topics-all":38,"news-related-fd9298b5-dea9-4641-8f73-56ae32261c3e":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"fd9298b5-dea9-4641-8f73-56ae32261c3e","扩散LM投机采样:串行评估砍到1\u002F4,分布零走样","扩散LM并行解码会偷换输出分布,精确采样又得逐token串行。arXiv 2611.08362 的 SpecCTMC 免训练、零草稿模型:复用已算预测当草稿、精确接受-校正,定理证明输出分布与直接仿真一致。LLaDA-8B 与 Dream-7B 五基准串行评估少 1.73-4.38 倍,B200 峰值提速2.88倍。","扩散语言模型(Diffusion LM)最诱人的能力,是一次能同时揭开多个 token——不像自回归模型只能从左到右逐字吐。但这份并行的诱惑里藏着一个坑:从同一个半遮半掩的序列里独立采样多个位置,token 之间的依赖会被打断,**你拿到的分布已经不是模型本来的分布了**。论文把这类误差称为 factorization error,此前更有工作直呼其为「并行解码的诅咒」。Fast-dLLM、Fast-dLLM++ 这类置信度并行解码器,以及 tau-leaping 数值采样器,快是快了,但都是用近似换速度。\n\n## 精确采样的代价:一个 token 一次串行评估\n\n换到连续时间视角,扩散 LM 的反向生成过程是一条连续时间马尔可夫链(CTMC):每个事件揭开一个 token。这种精确采样不引入离散化或分解误差,但代价是每个 token 都要一次串行的模型评估,上一个没算完,下一个不能开始。想要「快而不变味」,就得在这条链上做文章。\n\n## SpecCTMC:把投机解码搬进 CTMC\n\n[arXiv 2611.08362](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08362) 提出 SpecCTMC,思路对标自回归 LM 的投机解码(Leviathan\u002FChen, 2023),但有两处关键改造。其一,**免训练、不需要独立草稿模型**——草稿直接复用目标模型已经算过的预测。其二,并行评估若干候选中间态,再用精确的接受-校正程序裁决:第一个被拒绝的位置从正残差分布重采样修正,后续草稿全部丢弃、从修正态重新起草。整条链成立靠一个漂亮的观察:对 absorbing-mask 生成器,**事件时钟只依赖剩余掩码数量,与已揭开 token 的内容无关**,因此时钟可以提前采样,并在拒绝-校正中原样保留。定理 1 证明:SpecCTMC 输出轨迹(事件时间、生成顺序、token 取值)的联合分布与逐 token 直接仿真完全一致——既无时间离散化误差,也无并行分解误差,连 pass@k 都逐 k 相同。\n\n## 数字:评估省 1.73-4.38 倍,峰值提速 2.88 倍\n\n在 LLaDA-8B-Instruct 与 Dream-v0-Instruct-7B 两个模型、GSM8K\u002FMATH\u002FMBPP\u002FHumanEval\u002FIFEval 五个基准上,串行目标模型评估减少 1.73-4.38 倍;温度 1 的随机采样设定下,两模型平均准确率分别为 16.0→16.5 与 14.8→14.6,基本平价。NVIDIA B200 实测 wall-clock 平均提速 1.36-1.55 倍,IFEval 上峰值 2.88 倍;多模态 LLaDA-V 上收益更大,MathVerse\u002FMathVista 评估减少 5.93-6.69 倍。GSM8K 上做 K=8 次采样,SpecCTMC 用 724 次中位累计评估,达到与 256 步 tau-leaping(2056 次)相同的 71.88% Pass@8,省约 2.84 倍。[代码](https:\u002F\u002Fgithub.com\u002Fkasakh\u002Fspecctmc)以 Apache 2.0 开源。\n\n## 评论:诚实的天花板与真正的价值\n\n必须泼盆冷水:wall-clock 增益明显小于评估数降幅,因为并行 batch 不是免费的——长 prompt 的 GSM8K 在 Dream 上只打平甚至更慢,短序列、多分支的 HumanEval\u002FIFEval 才是大赢家。但方向价值很清楚:贪心解码永远只给一个答案,而多次随机采样能覆盖贪心漏掉的解;此前这条路被「精确=串行」锁死,SpecCTMC 把精确随机采样的成本压进了可负担区间。加速不应以偷换模型输出为代价——快而不变味,才是扩散 LM 走向生产的门票。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08362","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"9a32d315-c582-432b-978b-3d35aba21e02","en","SpecCTMC: exact speculative sampling for diffusion LMs","SpecCTMC brings distribution-preserving speculative decoding to masked diffusion LMs: 1.73-4.38x fewer sequential evals, up to 2.88x on B200.","Diffusion language models (diffusion LMs) promise something autoregressive models cannot: unmasking many tokens at once instead of writing strictly left to right. But that parallelism hides a trap. Independently sampling several positions from the same partially masked sequence breaks the dependencies between tokens — **the distribution you draw from is no longer the model's distribution**. The paper calls this factorization error, and earlier work dubbed it the \"curse of parallel decoding.\" Confidence-based parallel decoders such as Fast-dLLM and Fast-dLLM++, and numerical samplers like tau-leaping, are all trading exactness for speed.\n\n## The price of exactness: one sequential evaluation per token\n\nIn the continuous-time view, the reverse generation of a diffusion LM is a continuous-time Markov chain (CTMC): each event unmasks one token. This exact sampling introduces no discretization or factorization error, but it costs one sequential model evaluation per token — the next evaluation cannot start before the previous one finishes. Making it fast *without* changing the flavor is the real problem.\n\n## SpecCTMC: speculative decoding, ported to the CTMC\n\n[arXiv 2611.08362](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08362) introduces SpecCTMC, in the spirit of speculative decoding for autoregressive LMs (Leviathan\u002FChen, 2023), with two key twists. First, it is **training-free and needs no separate draft model** — drafts reuse predictions the target model has already computed. Second, it evaluates several candidate intermediate states in parallel, then applies an exact acceptance-and-correction procedure: at the first rejected position it resamples from the positive residual, discards all later tentative marks, and restarts drafting from the corrected state. The whole construction rests on an elegant observation: for absorbing-mask generators, **event clocks depend only on the number of remaining masks, not on which tokens have been revealed**, so the clocks can be sampled in advance and preserved through rejection and correction. Theorem 1 proves that the joint distribution of SpecCTMC's trajectory — event times, generation order, and token values — is identical to direct token-by-token simulation: no time-discretization error, no extra factorization error, and the same pass@k for every k.\n\n## The numbers: 1.73-4.38x fewer evaluations, up to 2.88x faster\n\nAcross LLaDA-8B-Instruct and Dream-v0-Instruct-7B on five benchmarks (GSM8K, MATH, MBPP, HumanEval, IFEval), SpecCTMC cuts sequential target-model evaluations by 1.73-4.38x. At temperature 1, average accuracy moves from 16.0 to 16.5 and from 14.8 to 14.6 on the two models — essentially at parity. On an NVIDIA B200, measured wall-clock speedups average 1.36-1.55x with a peak of 2.88x on IFEval; on the multimodal LLaDA-V the gains grow to 5.93-6.69x fewer evaluations on MathVerse and MathVista. On GSM8K with K=8 attempts, SpecCTMC reaches the same 71.88% Pass@8 as 256-step tau-leaping using 724 versus 2,056 median cumulative evaluations — roughly 2.84x cheaper at equal coverage. The [code](https:\u002F\u002Fgithub.com\u002Fkasakh\u002Fspecctmc) is open-sourced under Apache 2.0.\n\n## Commentary: honest ceilings and the real prize\n\nA dose of cold water: wall-clock gains trail the reduction in evaluations, because parallel batching is not free — long-prompt GSM8K on Dream is at parity or slower, while short-sequence, branch-heavy HumanEval and IFEval are the big winners. But the direction matters. Greedy decoding always yields a single answer, while repeated stochastic sampling can cover solutions greedy misses; that path used to be locked behind \"exact = sequential,\" and SpecCTMC pushes exact stochastic sampling into an affordable range. Speedups should not silently swap out a model's outputs — fast *and* faithful is the ticket for diffusion LMs to reach production.","specctmc-exact-speculative-diffusion-lm","2026-11-07T15:30:00Z","2026-11-07T15:08:17.934682Z","2026-11-07T15:08:17.934693Z",true,"agent",274,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"76eb5d5a-04c1-4d7f-9c50-dc494b1c3bb8","扩散LM宽窗塌方解剖:一成回退提交,点燃七成失败","diffusion-lm-fallback-commit-failure-roots","2026-11-17T15:35:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"269851b4-d6e6-43a6-9239-b3116708b735","块边界是同步税:扩散LM重叠去噪免训练提速14.5%","b3-overlapping-block-denoising-dlm","2026-11-11T17:11:59+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"51953ff7-5702-4d21-9af6-92d8edf7af09","块扩散LM双稀疏提速:不动权重,32K单步快4.72倍","duet-dual-sparse-bdlm-inference","2026-11-09T15:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"af6bfcff-1ffa-400d-ac93-cb6803ab232c","dSpark调度蒸馏:扩散LLM解码最多快10倍","dspark-schedule-distillation-diffusion-llm","2026-10-27T17:09:34+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"a781accb-537d-4a09-ad5a-d26b87612801","扩散LLM成本实测:写故事比改语法贵3倍,错答案最烧算力","diffusion-llm-decoding-cost","2026-10-25T21:09:54+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"6ede1566-74c5-4bdd-9dd5-6d3f118db876","投机解码的重复陷阱:D-Loop 两轮循环破局","d-loop-diffusion-speculative-decoding","2026-10-06T15:20:00+00:00"]