[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-eos-mismatch-opd-length-inflation":3,"topics-all":38,"news-related-28c41f06-d20f-481c-b133-cd109af3aed1":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"28c41f06-d20f-481c-b133-cd109af3aed1","答对之后停不下来:微软团队揪出在线蒸馏的 EOS 错配元凶","arXiv 新论文定位在线蒸馏长度膨胀元凶:师生 EOS 终止 token 错配会压制学生的停止动作,Qwen3 实例答对后仍多写 7,098 个 token;把等价 EOS 聚合为单一语义停止动作后,Gemma 回复长度从 7,168 预算饱和回落到约 2,000。","在线蒸馏(on-policy distillation,OPD)有个怪现象困扰着不少团队:学生模型明明答对了,却怎么也停不下来。9 月 17 日挂上 arXiv 的论文([arXiv:2609.20511](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.20511),微软联合 UNC 团队,30 页)给出意外定位——问题多半不在 reward、不在数据,而在几乎没人检查的细节:师生双方的 EOS 终止 token 对不上。\n\n## 症状:答案正确,然后狂写废 token\n\n一条 Qwen3 rollout 在第 1,094 个 token 就写出了正确答案,随后又生成 7,098 个冗余 token;另一条在第 462 个 token 答对,然后把 8,192 token 预算的 94% 花在重复标点上。答案是对的,评分是对的,但几乎整段生成都是浪费。\n\n且这不是个别样本,而是训练的收敛方向:200 次更新后,平均回复长度一路爬满 token 预算,被截断(从未主动终止)的回复比例接近 100%,换 prompt 模板也一样。\n\n## 病根:终止概率压在采不到的 token 上\n\n作者测量了症状对应的那个量:在学生实际停下的位置,它给「停下」分配了多少概率。Qwen 学生起初是学会终止的——第 35 步左右终止概率升到约 0.87,随后开始遗忘,到第 150 步,停下的概率与零无异,长度恰在此刻饱和。Llama 和 Gemma 则从未学会。\n\n机制上:base 学生把一个 token 当作终止符,而 post-trained 教师在另一个 token 上结束回合——Qwen 对里是 `\u003C|endoftext|>` 对 `\u003C|im_end|>`,Llama 对是 1 个终止 id 对 3 个。教师确信回合该结束,但这份置信落在 base 学生解码器永远采不出的 token 上——既不进 rollout、也拿不到梯度;学生能发出的终止 token 又得不到教师支持。「停」成了目标函数无法奖励的动作,只剩「继续」。\n\n## 修复:把「停」当一个语义动作\n\n对齐解码停止集本身是不够的——论文专门验证了这一点。真正起效的是把师生功能等价的 EOS token 聚合成一个语义停止动作来监督(`EOS_MODE=semantic_class`)。修复后长度膨胀直接反转:Gemma 从饱和 7,168 token 预算、约 100% 截断率,回落到均值约 2,000 token(接近教师在同批 prompt 上的中位数),截断近乎归零;Llama 降到约 3,500-4,000,截断率大约减半。该结论在 Qwen3、Llama 3.2、Gemma 3 三族复现,评测覆盖 AIME24\u002F25 与 AMC23。\n\n## 没修完的部分与社区反应\n\n修复并非全剧终:K2-Horizon 师生对的 400 步长程实验里,终止对齐后训练后期仍冒出一种独立的长度膨胀——错配是重要来源,但不是全部。代码已开源([github.com\u002FUNCSciML\u002Fopd-eos](https:\u002F\u002Fgithub.com\u002FUNCSciML\u002Fopd-eos),含 5 种 EOS 条件消融),论文页下有从业者感慨:追了半天的长度膨胀居然可能是 token id 错配而非 reward hacking,「我们一群人可能一直在 debug 错的东西」。\n\n对正在跑 OPD 的团队,论文给的启示是一套诊断顺序:遇到长度失控,先查师生终止 token 对齐(看教师在学生终止位置的概率分布),再怪目标函数、数据或训练 horizon。一个 token 级细节,决定一次蒸馏 run 的大半 token 开销与墙钟时间。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.20511","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"edd99705-63ca-4cf6-8041-b0c66332931f","en","EOS Token Mismatch: The Hidden Driver of OPD Length Inflation","New arXiv paper pins OPD length inflation on teacher-student EOS mismatch; one semantic-stop fix drops Gemma from the 7,168 budget ceiling to ~2,000 tokens.","A strange failure mode is troubling teams running on-policy distillation (OPD): the student model answers correctly, then simply cannot stop. A paper posted to arXiv on September 17 ([arXiv:2609.20511](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.20511), a Microsoft-UNC collaboration, 30 pages) offers a surprising diagnosis — the culprit is usually not the reward, not the data, but a detail almost nobody checks: the teacher's and student's EOS termination tokens disagree.\n\n## The symptom: correct answers, then thousands of wasted tokens\n\nThe paper opens with a vivid case. One Qwen3 rollout reaches the correct answer at token 1,094, then generates 7,098 redundant tokens; another hits the right result at token 462 and spends 94% of its 8,192-token budget repeating punctuation. The answer is right, the grade is right — and almost the entire generation is waste.\n\nNor is this a tail of unlucky samples; it is where training converges. Over 200 updates, mean response length climbs into the token budget and the fraction of clipped (never-terminated) responses approaches 100%, under both prompt templates tested.\n\n## The root cause: stopping probability lands on an unsampleable token\n\nThe authors measure the exact quantity the symptom is about: at the position where the student actually stopped, how much probability it places on stopping. The Qwen student does learn to terminate — stopping probability rises to roughly 0.87 around step 35 — then unlearns it; by step 150 the probability of stopping where it stopped is indistinguishable from zero, exactly when length saturates the budget. Llama and Gemma never get there at all.\n\nMechanistically: a base student treats one token as terminal, while its post-trained teacher ends assistant turns on a different one — `\u003C|endoftext|>` versus `\u003C|im_end|>` for the Qwen pair, one terminal id against three for Llama. The teacher is confident the turn is over, but that confidence sits on a token the base student's decoder can never sample, so it never appears in a rollout and never receives gradient; meanwhile the token the student can emit gets no support from the teacher. Terminating becomes an action the objective cannot reward, and continuing is all that remains.\n\n## The fix: treat stopping as one semantic action\n\nAligning the decoding stopping set alone is insufficient — the paper verifies this explicitly. What works is aggregating the student's and teacher's functionally equivalent EOS tokens into a single semantic stopping action, supervised with the teacher's summed terminal mass (`EOS_MODE=semantic_class`). Length inflation reverses with it: Gemma drops from saturating the 7,168-token budget at a ~100% clip rate to a mean of roughly 2,000 tokens — close to the teacher's own median on the same prompts — with clipping near zero; Llama comes down to ~3,500-4,000 tokens and roughly halves its clip rate. The finding reproduces across Qwen3, Llama 3.2, and Gemma 3, evaluated on AIME24\u002F25 and AMC23.\n\n## What remains unfixed, and the community reaction\n\nThe fix is not the whole story: in a 400-update long-horizon run on a K2-Horizon student-teacher pair, a distinct late-stage length inflation persists even after termination alignment — mismatch is an important source, but not the only one. The code is open-sourced ([github.com\u002FUNCSciML\u002Fopd-eos](https:\u002F\u002Fgithub.com\u002FUNCSciML\u002Fopd-eos), including a five-condition EOS ablation), and under the paper page a practitioner commented that the length inflation he had been chasing might be a token-id mismatch rather than reward hacking — \"a bunch of us have been debugging the wrong thing entirely.\"\n\nFor teams running OPD, the practical takeaway is a diagnostic order: when length runs out of control, check teacher-student termination-token alignment first (inspect the teacher's EOS distribution at the positions where the student terminates) before blaming the objective, the data, or the horizon. A token-level detail decides most of the token spend and wall-clock time of a distillation run.","eos-mismatch-opd-length-inflation","2026-09-18T21:09:06Z","2026-09-18T21:09:08.140148Z","2026-09-18T21:09:08.140162Z",true,"agent",316,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"58ed753e-ad6d-4aac-95f4-36bf217e169c","把 10 万条人类视频变成机器人教材:RoboTok 检索 mAP 提升约 50 倍,hard 任务 79.3% 对 19.5%","robotok-retrieval-benchmark-reread","2026-09-06T21:11:25+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"005557c5-8a3c-4d34-89bc-35d5351c4570","蒸馏只需要一条训练样本?清华实测:单条query覆盖71.5%训练状态,16条追平17k全量","one-shot-opd-single-query-distillation","2026-09-05T21:07:11+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"7623f190-7071-4811-a6f1-32462a99b8d3","经验会过期:阿里云论文让自主后训练的有害授权率从 62.5% 降到 25%","bcit-conditional-experience-transfer-post-training","2026-09-05T17:11:11+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"4a89fe5a-8703-49e5-b083-079cbda0fa2a","蒸馏也有副作用:中间训练期上KD,推理上涨、事实记忆反而变慢","switch-distillation-midtraining-kd","2026-09-02T17:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"3c9e4d7f-6f2a-4fea-8d22-c351b8fd7a4a","IBM Granite 4.1：Dense架构回归，8B参数挑战32B MoE性能","ibm-granite-4-1-dense-8b-moe-32b-grc","2026-04-29T19:10:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"12c67d52-17a2-4df5-8386-35d18ffd221a","JEPA-Anything:一套预测框架打通七个领域,湿实验也给了背书","jepa-anything-orthogonal-predictive-factorization","2026-09-19T23:10:37+00:00"]