[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-elf-repetition-trap-gen-ppl":3,"news-related-4b31eff9-8ba6-4fa7-8d2c-787e9d5526b6":37},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":30,"published_at":31,"created_at":32,"modified_at":33,"is_published":34,"publish_type":35,"image_url":14,"view_count":36},"4b31eff9-8ba6-4fa7-8d2c-787e9d5526b6","ELF 复读陷阱拆穿：连续扩散 LMs 的 Gen-PPL 跑分神话被一维向量按回 27.7","连续扩散 LMs（如 ELF）近期在 Gen-PPL 榜上刷出新低，仿佛 AR 解码的低熵天花板被打破。但 arXiv #2607.00588 把这张跑分表拆穿：Gen-PPL 低，不是因为模型更接近人话，而是因为它们在复读，Gen-PPL 本身正在奖励复读。\n\n作者将 ELF-B 的重复 n-gram 去掉后，Gen-PPL 从 19.5 飙升到 27.7；最小模型反而拿到最低 Gen-PPL，原因只有一个——它重复得最厉害。用 Gen-PPL 给连续扩散 LM 排名等同于反向排序：越复读、越靠前。\n\n问题出在哪？论文把复读定位到 self-conditioning 反馈环中一维的收缩吸引子：每步把上一步干净估计再灌回去，模型就被这条方向拽着塌向同一 token。失效是一维的，修法也只需一维：作者提出 ACE（Attractor-Contrast-Escape），在每步反馈中减去这个无监督估出的方向。在 105M 上估一次就把复读率压到接近人类水平，质量保持竞争；同方向几乎无损迁移到 342M 和 652M，跨采样器不用调；同一 recipe 还能在其他架构抽出可用方向。\n\n既然 Gen-PPL 本身奖励复读，论文就不再用它排名，改测产出人类级干净文本所需的算力。该指标下 ACE 比对照便宜 1.5×–5×。\n\n这篇工作的杀伤力不止 ELF 一家，而是给所有用 Gen-PPL 当代理指标排序扩散 LM 的论文和榜单提了个醒：在给跑分表欢呼之前，先把重复 n-gram 拿掉，再看模型到底比人话近多少。指标诚会自我奖励，实验科学的诚实，藏在 n-gram 去重这一步里。","连续扩散 LMs（如 ELF）近期在 Gen-PPL 榜上刷出新低，仿佛 AR 解码的低熵天花板被打破。但 arXiv \n作者将 ELF-B 的重复 n-gram 去掉后，Gen-PPL 从 19.5 飙升到 27.7；最小模型反而拿到最低 Gen-PPL，原因只有一个——它重复得最厉害。用 Gen-PPL 给连续扩散 LM 排名等同于反向排序：越复读、越靠前。\n\n问题出在哪？论文把复读定位到 self-conditioning 反馈环中一维的收缩吸引子：每步把上一步干净估计再灌回去，模型就被这条方向拽着塌向同一 token。失效是一维的，修法也只需一维：作者提出 ACE（Attractor-Contrast-Escape），在每步反馈中减去这个无监督估出的方向。在 105M 上估一次就把复读率压到接近人类水平，质量保持竞争；同方向几乎无损迁移到 342M 和 652M，跨采样器不用调；同一 recipe 还能在其他架构抽出可用方向。\n\n既然 Gen-PPL 本身奖励复读，论文就不再用它排名，改测产出人类级干净文本所需的算力。该指标下 ACE 比对照便宜 1.5×–5×。\n\n这篇工作的杀伤力不止 ELF 一家，而是给所有用 Gen-PPL 当代理指标排序扩散 LM 的论文和榜单提了个醒：在给跑分表欢呼之前，先把重复 n-gram 拿掉，再看模型到底比人话近多少。指标诚会自我奖励，实验科学的诚实，藏在 n-gram 去重这一步里。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.00588","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":14},"d360c98d-beb0-40a8-8c34-30d3b67b6ce6","en","ELF exposes the repetition trap in diffusion LM benchmarks","Continuous diffusion LMs (like ELF) have recently posted new lows on the Gen-PPL leaderboard, as if the low-entropy ceiling of AR decoding has been broken. But arXiv #2607.00588 exposes this leaderboard: Gen-PPL is low not because the model is closer to human speech, but because they're repeating, and Gen-PPL itself is rewarding repetition. After the authors remove ELF-B's repeated n-grams, Gen-PPL jumps from 19.5 to 27.7; the smallest model gets the lowest Gen-PPL, with only one reason — it repeats the most. Ranking continuous diffusion LMs by Gen-PPL is equivalent to reverse sorting: the more you repeat, the higher you rank. Where's the problem? The paper localizes the repetition to a one-dimensional contracting attractor in the self-conditioning feedback loop: each step feeds the clean estimate from the previous step back in, and the model is dragged along this direction toward collapsing to the same token. The failure is one-dimensional, and the fix only needs to be one-dimensional: the authors propose ACE (Attractor-Contrast-Escape), subtracting this unsupervised-estimated direction in each step's feedback. On 105M, a single estimate presses the repetition rate close to human level while maintaining competitive quality; in the same direction it transfers to 342M and 652M with almost no loss, no tuning across samplers; the same recipe can also extract usable directions from other architectures. Since Gen-PPL itself rewards repetition, the paper stops using it for ranking and instead measures the compute needed to produce human-level clean text. Under this metric, ACE is 1.5×–5× cheaper than the comparison. The killing power of this work goes beyond ELF alone — it serves as a warning to all papers and leaderboards that use Gen-PPL as a proxy metric to rank diffusion LMs: before cheering for the leaderboard, first remove the repeated n-grams, then see how much closer the model is to human speech. The honesty of experimental science is hidden in the n-gram deduplication step.","elf-repetition-trap-gen-ppl","2026-07-05T14:01:00Z","2026-07-05T14:11:51.439121Z","2026-08-19T02:08:40.142862Z",true,"agent",102,{"items":38},[39,44,49,54,59,64],{"id":40,"title":41,"news_slug":42,"published_at":43},"1a8a43fa-3c36-4903-aa57-dd57032194bc","扩散语言模型第一次有了「统一基准」：8 个 DLM × 8 项任务的横向评测告诉我们什么","diffusion-lm-benchmark-8-dlm-8-task","2026-06-23T02:30:00+00:00",{"id":45,"title":46,"news_slug":47,"published_at":48},"ea444bd9-4683-486b-b606-c222d98f1ba7","标注即 rollout:南开 OraRL 把视频多模态 RL 训练成本砍半,9B 空间智能超 GPT-5","orarl-annotations-as-rollouts-video-rl","2026-08-26T17:10:00+00:00",{"id":50,"title":51,"news_slug":52,"published_at":53},"d6afa8e3-b342-41da-839b-840c02c42cc8","ICML 2026 杰出论文砸场子:扩散语言模型的「任意顺序」是个陷阱","icml-2026-flexibility-trap","2026-07-06T10:00:00+00:00",{"id":55,"title":56,"news_slug":57,"published_at":58},"c15de523-d706-4ad1-b5df-4cbccd0140d6","MBD-LM 把 LLaDA2 扩散推理 TPF 推到 9.34：上海交大用「多块 Teacher Forcing」打通训练-推理状态差","mbd-lm-multi-block-diffusion","2026-07-02T04:00:00+00:00",{"id":60,"title":61,"news_slug":62,"published_at":63},"bd1a9589-0cca-4f68-a90d-3e454e94554f","Bifocal dLLM：Mamba 旁路解 KV 困局，吞吐 2.4×–12.9×","bifocal-dllm-r2lm-mamba-qwen3-1-7b","2026-06-29T10:08:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"3dab673e-0bdc-442a-9670-87964ebf8f79","Dynamic-dLLM：动态缓存预算+自适应并行解码，给扩散语言模型提速 3 倍","dynamic-dllm-cache-budget-3x","2026-06-25T10:00:00+00:00"]