[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-dspark-schedule-distillation-diffusion-llm":3,"topics-all":38,"news-related-af6bfcff-1ffa-400d-ac93-cb6803ab232c":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"af6bfcff-1ffa-400d-ac93-cb6803ab232c","dSpark调度蒸馏:扩散LLM解码最多快10倍","扩散语言模型能并行出token,却要很多轮解码才收尾。MBZUAI团队提出dSpark:先揭\"不确定\"的token,让其余位置更早可预测。冻结教师排出揭示顺序再蒸馏给学生模型,GSM8K上LLaDA-8B前向压缩10.12倍、Dream-7B压缩5.12倍,精度持平或更好(论文自报);权重尚未放出。","扩散语言模型(dLLM)有个出名的不对称:一轮前向能同时预测很多 token,却要反复解码很多轮才能真正收尾。并行度有了,轮次下不去,速度还是慢。MBZUAI VILA Lab 的 Aidar Myrzakhan 与 Zhiqiang Shen 在 10 月 26 日提交到 arXiv 的论文里给出一个反直觉的答案:瓶颈不在\"哪些 token 好猜\",而在\"先揭开哪一个\"。论文提出的框架叫 dSpark,核心是把\"揭示顺序\"本身当成可学习、可蒸馏的知识。\n\n## 反直觉发现:最自信的 token 往往最不值得先揭\n\n主流的置信度解码策略,是先提交模型最有把握的 token,把没把握的留到后面。dSpark 的测量恰好指向反面:在 LLaDA 的正确 GSM8K 回答上,先揭开一个\"不确定\"的 token(置信度解码会推迟揭示的那类),能让其余被遮位置的 surprisal 多降 4.7 倍,并让多 5.1 倍的位置提前\"就绪\"。换句话说,最可预测的 token 常常提供最少的上下文;先揭难的那块,拼图反而成形更快。论文还观察到:不确定性区域越大的回答,需要的前向次数越多——轮次成本直接挂在\"没揭开的信息量\"上。\n\n## 方法:冻结教师排出新顺序,再蒸馏给学生\n\ndSpark 分两步。第一步,让冻结的教师模型对同一条正确回答换一种顺序揭示:每一轮挑\"最能降低其余位置不确定性\"的那个 token 先揭。第二步,把这些揭示轨迹蒸馏进学生模型,让模型自己学会这种调度;同时在模型自己的失败回答上加 KL 锚定,防止快速解码把生成质量带崩——README 的说法是,正确轨迹教调度,失败轨迹做正则。\n\n训练数据全部自产:基座模型在 GSM8K、PRM12K、NuminaMath-CoT、AceCode 上生成回答(Dream 基座另加 OpenMathInstruct-2),最多 256 个新 token、block size 32,按参考答案或单元测试筛出正确与失败两类轨迹,分别喂给调度学习与 KL 正则。仓库为 LLaDA 与 Dream 各建了目录,提供从数据构造、训练到评测的脚本,快速上手示例直接加载合并后的 dSpark-LLaDA checkpoint 做推理。\n\n## 数字与边界\n\n按论文报告,GSM8K 上 dSpark 把 LLaDA-8B-Instruct 的前向次数压缩 10.12 倍(tokens per forward 达到 10.12),把 Dream-7B-Instruct 压缩 5.12 倍,同时精度持平或更好;评测覆盖 GSM8K、MATH、HumanEval、MBPP,基于 lm-evaluation-harness,除准确率外还报告 tokens-per-forward 解码速度。这些数字都是作者自报,目前没有第三方复现。\n\n冷水也要泼:仓库明说模型权重与训练数据\"即将放出\",写稿时还没放出;实验集中在数学与代码两类任务、两个 8B 级开源基座;训练配置里最多 256 个新 token,长回答场景未经验证。\n\n## 所以呢\n\ndLLM 的加速竞赛正在换轴:上一阶段比\"猜得更准\",用更少轮次收敛;这一阶段开始比\"调度更聪明\"——同一个模型,只换\"先揭谁\",前向次数就能差出一个数量级(论文口径)。对做推理引擎的人来说,这比又刷高一个 benchmark 更值得盯:调度是零参数改动的加速杠杆,而且是能蒸馏、能迁移的杠杆。当然,在权重放出、有人在新任务上复现之前,\"10 倍\"请先按论文口径理解。[论文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.35804) \u002F [代码](https:\u002F\u002Fgithub.com\u002FVILA-Lab\u002FdSpark)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.35804","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"88998ed5-1ab3-46ef-93d1-c570b7f14cc0","en","dSpark Speeds Up Diffusion LLMs by Learning When to Commit","Schedule distillation for diffusion LLMs: dSpark reveals uncertain tokens first, cutting GSM8K forward passes 10.12x (LLaDA-8B) and 5.12x (Dream-7B).","Diffusion language models have a well-known asymmetry: a single forward pass can predict many tokens at once, yet the model needs many decoding rounds before a response actually finishes. The parallelism is there; the round count is what kills latency. A paper submitted to arXiv on October 26 by Aidar Myrzakhan and Zhiqiang Shen of VILA Lab, MBZUAI, argues that the bottleneck is not which tokens are easy to predict, but which one you reveal first. The proposed framework, dSpark, treats the reveal order itself as knowledge that can be learned and distilled.\n\n## The counterintuitive finding: the most confident token is often the least worth revealing first\n\nConfidence-based decoders commit the tokens the model is most sure about and defer the uncertain ones. The measurements here point the other way: on correct GSM8K responses from LLaDA, revealing an \"uncertain\" token — the kind confidence-based decoders would defer — lowers the surprisal of the other masked positions 4.7x more and makes 5.1x more of them ready, compared with revealing a confident one. The most predictable token often provides the least context; reveal the hard piece first and the puzzle assembles faster. The paper also observes that responses with a larger uncertainty area need more forward passes — round cost hangs directly on how much information is still unrevealed.\n\n## Method: a frozen teacher ranks a new reveal order, then distills it into the student\n\ndSpark works in two steps. First, a frozen teacher re-reveals the same correct response in a new order: each round it promotes the token that most reduces the uncertainty of the remaining masked positions. Second, those promoted trajectories are distilled into the student so the model learns the schedule itself, with a KL anchor on the model's own failed responses to keep fast decoding from wrecking quality — in the README's phrasing, correct trajectories teach the schedule while failed ones serve as the regularizer.\n\nTraining data is fully self-generated: the base model produces responses on GSM8K, PRM12K, NuminaMath-CoT and AceCode (plus OpenMathInstruct-2 for the Dream base), with at most 256 new tokens and a block size of 32; responses are checked against reference answers or unit tests and split into correct and failed trajectories for schedule learning and KL regularization respectively.\n\n## Numbers and caveats\n\nPer the paper's own reports, dSpark cuts forward passes on GSM8K by 10.12x on LLaDA-8B-Instruct (10.12 tokens per forward) and by 5.12x on Dream-7B-Instruct, while matching or improving accuracy. Evaluation covers GSM8K, MATH, HumanEval and MBPP via lm-evaluation-harness, reporting accuracy plus tokens-per-forward decoding speed. These are author-reported numbers with no third-party replication yet.\n\nThe cold water: the repo states that model checkpoints and training data \"will be released soon\" — at the time of writing they are not out; experiments concentrate on math and code tasks with two open 8B-class base models; the training config caps responses at 256 new tokens, so long-form generation is untested.\n\n## So what\n\nThe dLLM speed race is switching axes. The last phase competed on predicting better — converging in fewer rounds. This one competes on scheduling smarter: same model, change only \"who gets revealed first\", and forward passes differ by an order of magnitude (paper-reported). For inference-engine builders that is worth more attention than another benchmark point — scheduling is a zero-parameter acceleration lever, and one that can be distilled and transferred. Until the weights ship and someone replicates on fresh tasks, read \"10x\" as the paper's own claim. [Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.35804) \u002F [Code](https:\u002F\u002Fgithub.com\u002FVILA-Lab\u002FdSpark)","dspark-schedule-distillation-diffusion-llm","2026-10-27T17:09:34Z","2026-10-27T17:09:40.591917Z","2026-10-27T17:09:40.591928Z",true,"agent",405,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"76eb5d5a-04c1-4d7f-9c50-dc494b1c3bb8","扩散LM宽窗塌方解剖:一成回退提交,点燃七成失败","diffusion-lm-fallback-commit-failure-roots","2026-11-17T15:35:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"269851b4-d6e6-43a6-9239-b3116708b735","块边界是同步税:扩散LM重叠去噪免训练提速14.5%","b3-overlapping-block-denoising-dlm","2026-11-11T17:11:59+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"51953ff7-5702-4d21-9af6-92d8edf7af09","块扩散LM双稀疏提速:不动权重,32K单步快4.72倍","duet-dual-sparse-bdlm-inference","2026-11-09T15:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"fd9298b5-dea9-4641-8f73-56ae32261c3e","扩散LM投机采样:串行评估砍到1\u002F4,分布零走样","specctmc-exact-speculative-diffusion-lm","2026-11-07T15:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"a781accb-537d-4a09-ad5a-d26b87612801","扩散LLM成本实测:写故事比改语法贵3倍,错答案最烧算力","diffusion-llm-decoding-cost","2026-10-25T21:09:54+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"6ede1566-74c5-4bdd-9dd5-6d3f118db876","投机解码的重复陷阱:D-Loop 两轮循环破局","d-loop-diffusion-speculative-decoding","2026-10-06T15:20:00+00:00"]