[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-diffusion-lm-block-regeneration-verifier":3,"topics-all":38,"news-related-0eee562a-f5b8-4bb6-9842-5b6c63ef72ad":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"0eee562a-f5b8-4bb6-9842-5b6c63ef72ad","找得到却改不动:扩散语言模型的纠错要整块重写","扩散语言模型能重访已生成的文本,改错却很难:数独诊断显示,错误藏在隐藏状态里,重开全部错误格,77 个盘面只救回 6 个。DiVR 用共享骨干、约 1% 参数的验证器按最终对错打分、整块重写,把 LLaDA-8B 微调版五基准平均分从 35.3% 提到 43.4%,算力约为搜索基线一半,还能叠加 SFT 与 RL。","数独是研究「模型改错」的干净解剖台:每格只有一个正确数字,对错一目了然。论文作者让在数独轨迹上微调过一轮的 LLaDA-8B-Instruct 解 190 道唯一解 4×4 数独,然后做了个残酷实验:在 77 个失败盘面里,把写错的格子全部重新罩上 mask 让模型重填,只救回 6 个——连「哪格错了」都告诉它,修补率仍不到一成。更吊诡的是,错误明明写在模型内部:线性探针读末层隐藏状态,区分错误格与正确格的 AUROC 达 0.838,token 置信度只有 0.685。找得到、改不动,这是 masked 扩散语言模型(MDLM)纠错的真实处境,也是 [arXiv:2611.17320](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17320) 这篇新论文的出发点。\n\n## 定点修补为什么失败\n\n两个诊断结果都指向上下文在「护错」。其一,单独 mask 掉一个已提交的错误格让模型重填,原来那个错误数字的平均概率还有 0.869;提交十六步之后才重开,概率反而涨到 0.962——周围没变的格子仍在合力支持同一个错误。其二,把出错之后的推理一并回滚,最好的固定深度只修好 13 个盘面;甚至允许每题按金标准答案挑最优回滚深度,也只有 15 个。作者的判断很直白:错误信息可以从模型状态读出,但在原上下文里重填错误位置基本修不好;该做的是把一个区域整体重写,再用「最终能不能做对」来挑。\n\n## DiVR:共享骨干的裁判\n\n于是有了 DiVR(Diffusion Verifier for Regeneration)。它复用生成器的冻结骨干,外挂 rank-32 LoRA 适配器加两层 MLP 头,可训练参数约占骨干的 1%。训练信号只要最终答案对错:每题采 4 条生成器自己的轨迹(温度 0.7),2,000 道 GSM8K 训练题即可,不需要 token 级标注。解码时,生成器每写完一个 32-token 块,验证器就给画布打一次分;低于阈值 0.5 且每题三次的重生成预算没用完,就把整块重新 mask、采 4 个替代块,谁分高留谁,都不行就保留原块。画布 256 token(MATH-500 用 512),每块 16 步。生成器本身一个参数不动。\n\n## 论文实测的账本\n\n五个基准(GSM8K、MATH-500、Countdown、HumanEval、MBPP)上,LLaDA-8B-Base 微调版贪心平均 35.3%,DiVR 引导解码提到 43.4%,代价是贪心的 1.9–3.5 倍;同一张表里,搜索基线 SR-SMC 用 3.8–4.0 倍算力拿 39.1%,Prism 用 5.2–7.4 倍拿 37.3%。折算下来,引导解码约用 SR-SMC 七成计算量、不到 Prism 一半。单看 GSM8K:68.7% 到 78.8% 只花 2.1 倍,对照 SR-SMC 72.6%(4.0 倍)、Prism 71.9%(5.9 倍)。换生成器也成立:LLaDA-8B-Instruct 平均 39.4% 到 44.8%,best-of-4 模式到 49.8%;Dream-v0-Instruct-7B 从 45.0% 到 54.8%,Countdown 上 18.0% 到 37.2%,论文原文用的词是翻倍。泛化同样在:只在 GSM8K 训过的验证器,把 MATH-500 从 24.6% 拉到 33.8%;SFT 版验证器直接给未微调 Base 用,平均 29.3% 到 38.0%,比 SFT 本身的 35.3% 还高。它还能叠加后训练:diffu-GRPO 强化学习后平均 40.1%,再加引导到 45.4%。论文还记录了一个耐人寻味的现象:RL 把奖励里的正确项从 0.76 推到 1.35(满分 2),贪心准确率却几乎没动——RL 改变模型写什么,验证器决定哪块该扔,两份增益相加。\n\n## 管用的是裁判,不是更多采样\n\n固定重生成预算、只换「何时重写」的信号:承诺置信度做判据是 72.2%,模型自问 Yes\u002FNo 是 67.9%,去掉 LoRA 只留头是 74.2%,完整 DiVR 78.8%,成本 ×2.10,低于对照信号的 2.45–3.62。挑完整答案时,平均对数概率 68.5%,几乎等于随机挑(68.7%);majority vote 75.7%,DiVR 82.2%,四条采样的 oracle 上限 85.4%。修复还集中在开头:GSM8K 的 154 个修复案例,74% 从第 0 块出发,92% 落在前两块;第一块分数对「最终会错」的检出率 76%、误报 22%,Countdown 更是 99% 对 1%。阈值 θ 在 0.1 到 0.9 间拨动,准确率只在 77.5–79.2% 波动、成本 1.79–2.61 倍——它更像成本旋钮,不是效果开关。\n\n## 冷静的部分\n\n论文自带的限定值得照抄:主运行 78.8%,只换验证器训练种子是 77.5%,只换重解码种子是 77.6%,作者注明这是敏感性检查、不是端到端独立复现;outcome 监督依赖答案可校验的领域,数学与代码天然合适,开放域得另想办法。结论句可以概括为:对 MDLM 而言,修订更缺的是一个「何时从头再来」的学习判据,而不是更精的找错方法。这与自回归侧用最终对错训验证器、再拿它挑候选的老路子一脉相承,DiVR 把它搬到了部分 mask 的画布上。代码仓库在论文注脚给出:[minhaeoh\u002FDiVR](https:\u002F\u002Fgithub.com\u002Fminhaeoh\u002FDiVR)。\n\n模型知道自己在犯错,只是改不动自己。让一个共享骨干、只占 1% 参数的裁判喊「这块重来」,比让模型盯着自己的置信度反复犹豫,更便宜也更有效——对做扩散语言模型推理的人来说,这大概是论文里可直接抄走的一招。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17320","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"c4d77ea9-92f6-4805-b223-3f9abcaf6a5b","en","Found yet Unfixed: DiVR Rewrites Blocks in Diffusion LMs","DiVR scores blocks of a diffusion LM by final correctness and regenerates weak ones, lifting LLaDA-8B from 35.3% to 43.4% on five benchmarks.","Sudoku is a clean testbed for self-correction: every cell has exactly one correct digit, so right and wrong are unambiguous. The authors decoded 4x4 Sudoku puzzles with LLaDA-8B-Instruct after one epoch of supervised training on Sudoku propagation traces, then ran a brutal experiment: on 77 failed boards they re-masked every wrong cell and let the model refill them. Only 6 boards recovered — even with the exact error locations handed over, the repair rate stayed under one in ten. The stranger part is that the error is plainly visible inside the model: a linear probe on final-layer hidden states separates wrong cells from correct ones at AUROC 0.838, while token confidence manages only 0.685. Errors can be found but not fixed in place — that is the real situation for masked diffusion language models (MDLMs), and the starting point of the new paper [arXiv:2611.17320](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17320).\n\n## Why in-place repair fails\n\nBoth diagnostic findings point at the context defending the mistake. First, when a single committed wrong cell is masked again for refill, the original wrong digit keeps a mean probability of 0.869; re-opened sixteen steps after its commit, the probability rises to 0.962 — the unchanged surrounding cells keep supporting the same error. Second, rolling back the reasoning written after the error as well, the best fixed rollback depth repairs only 13 boards; even the best per-puzzle depth chosen with the gold answer reaches 15. The authors' verdict is blunt: error information is readable from the model's states, but re-predicting erroneous positions under their context rarely repairs them. The right move is to regenerate a region jointly and judge the alternatives by their eventual outcome.\n\n## DiVR: a verifier inside the generator\n\nHence DiVR (Diffusion Verifier for Regeneration). It reuses the generator's frozen backbone with a rank-32 LoRA adapter and a two-layer MLP head; trainable parameters are about 1% of the backbone. The training signal is only final-answer correctness: four sampled trajectories per question (temperature 0.7) over 2,000 GSM8K training questions, with no token-level labels. During decoding, whenever the generator finishes a 32-token block, the verifier scores the canvas; if the score falls below the 0.5 threshold and the budget of three regenerations per problem is not exhausted, the whole block is re-masked, four alternatives are sampled, and the best-scoring one is kept — otherwise the original stays. Canvases hold 256 tokens (512 for MATH-500), with 16 steps per block. The generator itself is never touched.\n\n## The ledger, as measured in the paper\n\nAcross five benchmarks (GSM8K, MATH-500, Countdown, HumanEval, MBPP), the fine-tuned LLaDA-8B-Base reaches 35.3% average with greedy decoding and 43.4% with DiVR-guided decoding at 1.9–3.5x the cost; in the same table the search baseline SR-SMC gets 39.1% at 3.8–4.0x and Prism 37.3% at 5.2–7.4x. Guided decoding thus uses about 70% of SR-SMC's compute and less than half of Prism's. On GSM8K alone, going from 68.7% to 78.8% costs 2.1x, against SR-SMC's 72.6% at 4.0x and Prism's 71.9% at 5.9x. It carries across generators: LLaDA-8B-Instruct goes from 39.4% to 44.8% (best-of-4 mode: 49.8%); Dream-v0-Instruct-7B from 45.0% to 54.8%, with Countdown at 18.0% to 37.2%, which the paper describes as a doubling. Generalization holds too: a verifier trained only on GSM8K lifts MATH-500 from 24.6% to 33.8%; the SFT generator's verifiers applied to the untuned Base raise its average from 29.3% to 38.0%, above the 35.3% of fine-tuning itself. DiVR also stacks with post-training: after diffu-GRPO reinforcement learning the average is 40.1%, and guidance takes it to 45.4%. The paper records an instructive detail: RL pushed the reward's correctness term from 0.76 to 1.35 out of a possible 2 while greedy accuracy barely moved — RL changes what the generator writes, the verifier decides which block to discard, and the two gains add.\n\n## The judge matters, not the sampling\n\nHolding the regeneration budget fixed and swapping only the signal: commit-confidence gets 72.2%, prompted self Yes\u002FNo 67.9%, a head-only verifier 74.2%, and full DiVR 78.8% at ×2.10, below the 2.45–3.62 of the control signals. When selecting among four finished answers, mean log-probability reaches 68.5%, nearly the same as picking at random (68.7%); majority vote gets 75.7%, DiVR 82.2%, with the oracle ceiling at 85.4%. Repairs cluster at the start: of GSM8K's 154 repaired problems, 74% depart at block 0 and 92% within two blocks; the block-0 score flags 76% of eventually wrong solutions with 22% false positives, and Countdown sits at 99% versus 1%. Sweeping the threshold from 0.1 to 0.9 moves accuracy only within 77.5–79.2% while cost runs 1.79–2.61x — the threshold is mainly a cost knob, not an effect switch.\n\n## The caveats\n\nThe paper's own limits deserve quoting: the primary run hits 78.8%, changing only the verifier-training seed gives 77.5% and changing only the re-decoding seed 77.6%, which the authors call sensitivity checks rather than independent end-to-end replications; outcome supervision depends on answer-checkable domains — math and code qualify naturally, open-ended tasks need something else. The conclusion can be paraphrased as: what revision in MDLMs lacks is a learned criterion for when to start over, more than a finer way to find the error. That continues the autoregressive tradition of training verifiers on final-answer correctness and using them to pick candidates; DiVR moves it onto the partially masked canvas. The code repository is linked in the paper footnote: [minhaeoh\u002FDiVR](https:\u002F\u002Fgithub.com\u002Fminhaeoh\u002FDiVR).\n\nThe model knows it is wrong; it just cannot fix itself. Letting a judge that shares the backbone and trains about 1% of its parameters call for a redo is cheaper and more effective than letting the model stare at its own confidence — for anyone doing diffusion-LM inference, that is the takeaway to steal.","diffusion-lm-block-regeneration-verifier","2026-11-13T19:45:00Z","2026-11-13T19:07:00.217743Z","2026-11-13T19:07:00.217759Z",true,"agent",177,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"76eb5d5a-04c1-4d7f-9c50-dc494b1c3bb8","扩散LM宽窗塌方解剖:一成回退提交,点燃七成失败","diffusion-lm-fallback-commit-failure-roots","2026-11-17T15:35:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"62681ce7-8a51-41a2-925b-d02ceec45749","快思考模型大考:能力分领先,可靠性第七","so-arena-system-one-reliability","2026-11-14T15:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"269851b4-d6e6-43a6-9239-b3116708b735","块边界是同步税:扩散LM重叠去噪免训练提速14.5%","b3-overlapping-block-denoising-dlm","2026-11-11T17:11:59+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"51953ff7-5702-4d21-9af6-92d8edf7af09","块扩散LM双稀疏提速:不动权重,32K单步快4.72倍","duet-dual-sparse-bdlm-inference","2026-11-09T15:20:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"fd9298b5-dea9-4641-8f73-56ae32261c3e","扩散LM投机采样:串行评估砍到1\u002F4,分布零走样","specctmc-exact-speculative-diffusion-lm","2026-11-07T15:30:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"a43599f1-d635-4174-9a25-6e1031c497b1","长上下文崩掉的真凶不是长度，是内容竞争","context-rot-attention-competition-not-length","2026-11-04T09:00:00+00:00"]