[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-tgopd-teacher-gated-on-policy-distillation":3,"topics-all":35,"news-related-9825e20d-c9eb-4300-99b5-12eb7d0e755d":54},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"9825e20d-c9eb-4300-99b5-12eb7d0e755d","自信的错误教师最危险:TGOPD 给在线蒸馏装提示级门控,教师 GPU 利用率 9.8% 升至 78.9%","在线蒸馏让冻结教师批改学生自己的输出,但教师并非在每个提示上都可靠。TGOPD 论文提出提示级可靠性门控:探针通过走稠密蒸馏,不通过转 verifier-grounded GRPO;论文报告 4B\u002F35B 学生六个单域设置全胜 Vanilla OPD,教师节点 GPU 利用率从 9.8% 提到 78.9%。","在线蒸馏(On-Policy Distillation,OPD)正在成为后训练阶段的流行加速手段:冻结的教师模型在学生自己生成的轨迹上提供 token 级稠密监督,比只看最终对错的稀疏信号高效得多。但 9 月 2 日提交到 arXiv、9 月 8 日登上 Hugging Face Daily Papers 的 TGOPD 论文,点破了一个被普遍忽视的盲点:vanilla OPD 对所有提示一视同仁地施加教师监督,从不检查教师在这个提示上究竟靠不靠谱。\n\n## 自信的错误教师,比没有教师更糟\n\n论文的核心观察基于一个数学性质:OPD 常用 reverse KL 作为目标,而 reverse KL 是 mode-seeking 的——一个「自信但错误」的教师会诱发强而误导的参数更新。熵、师生似然一致性这类分布式代理指标只能度量不确定性或一致性,并不能直接验证结果对错:它们能识别教师「不确定」,却识别不出教师「笃定地错」。\n\n## 提示级门控怎么建\n\nTGOPD(Teacher-Gated On-Policy Distillation)把做法概括成「先验证,再蒸馏」:先用一组经过验证器打分的教师探针,估计教师在当前提示上的可靠性;通过检查的提示走稠密 OPD,未通过的提示改走 verifier-grounded GRPO,两条路径互斥。这样一来,「教师是否可信」从默认假设变成了显式检查项。\n\n## 两个层面的收益\n\n实验覆盖 4B 与 35B 两档学生模型,横跨数学、代码、指令遵循三个域。论文报告:单域训练的全部六个设置中 TGOPD 均优于 Vanilla OPD;多域训练下,两个规模的学生都拿到更高的七基准平均分。更实际的是算力账——异步 OPD 里教师节点大量时间在闲置,把可靠性估计安排进这段空闲容量,实测 4B 单域训练中教师节点 GPU 利用率从 9.8% 提升到 78.9%,门控几乎是免费的。\n\n## 所以呢\n\n这篇论文真正值得带走的不是一个新训练技巧,而是一条工程原则:稠密监督的准入应当有验证条件。信号越密,越要问一句来源是否可靠——对蒸馏如此,对 RLHF 里的奖励模型、Agent 工作流里的工具反馈同样如此。「可靠走蒸馏、不可靠走强化学习」这种按可信度路由监督源的思路,大概率会在更多后训练管线里复现。论文原文见 [arXiv:2609.02998](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.02998)。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.02998","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"d5f84ca8-7c4d-4a9c-a44b-4df51a651a27","en","TGOPD Gates On-Policy Distillation by Teacher Reliability per Prompt","TGOPD verifies teacher reliability per prompt in on-policy distillation, routing trusted prompts to dense OPD and the rest to verifier-grounded GRPO.","On-policy distillation (OPD) has become a popular way to accelerate post-training: a frozen teacher provides dense token-level supervision on the student's own rollouts, far more efficient than sparse outcome-only signals. But a paper submitted to arXiv on Sep 2 and trending on Hugging Face Daily Papers on Sep 8 points out a widely ignored blind spot: vanilla OPD applies teacher supervision uniformly across prompts, without ever checking whether the teacher is actually reliable on the prompt at hand.\n\n## A Confidently Wrong Teacher Is Worse Than No Teacher\n\nThe core observation rests on a mathematical property: OPD typically uses reverse KL as its objective, and reverse KL is mode-seeking — a confidently wrong teacher induces a strong yet misleading update. Distributional proxies such as entropy or teacher-student likelihood agreement measure uncertainty or agreement, but they do not directly verify outcome correctness. They can catch an uncertain teacher; they cannot catch one that is confidently wrong.\n\n## How the Prompt-Level Gate Works\n\nTGOPD (Teacher-Gated On-Policy Distillation) compresses its recipe into four words: verify before you distill. It estimates teacher reliability on the current prompt from a small set of verifier-scored teacher probes. Prompts that pass the check go to dense OPD; prompts that fail are routed to verifier-grounded GRPO instead — the two paths are mutually exclusive. Teacher trust turns from a default assumption into an explicit checkpoint.\n\n## Gains on Two Levels\n\nThe experiments cover 4B and 35B students across mathematics, code, and instruction following. The paper reports that TGOPD outperforms Vanilla OPD in all six single-domain settings, and achieves higher seven-benchmark averages at both scales under multi-domain training. The more practical number is about compute: in asynchronous OPD the teacher node sits idle much of the time, and packing reliability estimation into that otherwise-idle capacity raised teacher-node GPU utilization from 9.8% to 78.9% in the measured 4B single-domain run — the gate is essentially free.\n\n## So What\n\nThe takeaway is not another training trick but an engineering principle: dense supervision should be admitted on verification. The denser the signal, the more you should ask whether its source is reliable — true for distillation, equally true for reward models in RLHF and tool feedback in agent workflows. Routing supervision by trustworthiness (distill when reliable, reinforce when not) is a pattern likely to reappear across post-training pipelines. The paper is at [arXiv:2609.02998](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.02998).","tgopd-teacher-gated-on-policy-distillation","2026-09-08T23:10:00Z","2026-09-08T23:05:14.144617Z","2026-09-08T23:05:14.144632Z",true,"agent",120,[36,45],{"slug":37,"tag_slug":37,"title_zh":38,"title_en":39,"intro_zh":40,"intro_en":41,"id":42,"is_active":32,"created_at":43,"modified_at":44},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":46,"tag_slug":46,"title_zh":47,"title_en":48,"intro_zh":49,"intro_en":50,"id":51,"is_active":32,"created_at":52,"modified_at":53},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":55},[56,61,66,71,76,81],{"id":57,"title":58,"news_slug":59,"published_at":60},"58ed753e-ad6d-4aac-95f4-36bf217e169c","把 10 万条人类视频变成机器人教材:RoboTok 检索 mAP 提升约 50 倍,hard 任务 79.3% 对 19.5%","robotok-retrieval-benchmark-reread","2026-09-06T21:11:25+00:00",{"id":62,"title":63,"news_slug":64,"published_at":65},"005557c5-8a3c-4d34-89bc-35d5351c4570","蒸馏只需要一条训练样本?清华实测:单条query覆盖71.5%训练状态,16条追平17k全量","one-shot-opd-single-query-distillation","2026-09-05T21:07:11+00:00",{"id":67,"title":68,"news_slug":69,"published_at":70},"7623f190-7071-4811-a6f1-32462a99b8d3","经验会过期:阿里云论文让自主后训练的有害授权率从 62.5% 降到 25%","bcit-conditional-experience-transfer-post-training","2026-09-05T17:11:11+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"4a89fe5a-8703-49e5-b083-079cbda0fa2a","蒸馏也有副作用:中间训练期上KD,推理上涨、事实记忆反而变慢","switch-distillation-midtraining-kd","2026-09-02T17:10:00+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"86c380ed-bdb5-47d0-bf9a-3c55f8573d61","on-policy 蒸馏真的在蒸馏吗?普渡论文:固定负优势就能追平教师","on-policy-distillation-teacher-free-opsa","2026-09-01T15:05:00+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"217f417d-1b9c-475b-99f4-e21e7c909711","MHAR 把 Transformer 残差流从「单车道」拆成 H 条独立路由:子空间第一次有权自己挑历史层","multi-head-attention-residuals-mhar","2026-08-01T07:30:00+00:00"]