[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-mopd-multi-teacher-on-policy-distillation":3,"topics-all":33,"news-related-0bf07f31-ca28-49f1-b296-c1b5e56d200d":52},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":20,"news_slug":26,"published_at":27,"created_at":28,"modified_at":29,"is_published":30,"publish_type":31,"image_url":13,"view_count":32},"0bf07f31-ca28-49f1-b296-c1b5e56d200d","MOPD 把多领域 RL 教师塞进同一个学生：小米\u002F北大发布的能力集成 post-training 新范式","背景：现代大模型 post-training 几乎都依赖强化学习来压出特定能力,但把代码、数学、推理、对话等需求同时塞到同一个模型里顾此失彼。现有的 Off-Policy Finetune、Mix-RL、Cascade RL、Param-Merge 等方案要么效率低,要么在某个领域掉点,权重和数据耦合太深,工程上难并行推进。\n\n核心方法：小米 LLM-Core 团队联合北大发表在 arXiv:2606.30406 的 MOPD(Multi-Teacher On-Policy Distillation)另辟蹊径,先对每个领域独立跑 RL 得到一组领域教师,然后让学生在自己产生的 rollouts 上,用 token 级 stop-gradient 的蒸馏优势 46804 Â_t = sg[\\log \\pi_{teacher}(t) - \\log \\pi_{student}(t)] 46804 反向更新。这一招同时解决两件事:消除 exposure bias(学生学的是自己真实分布下 rollouts 上遇到的 token,不再是离线数据集上分布外的样本);保留密集优化信号(token 级差异即时可用,不必等整个 episode 拿回报)。\n\n数字结果:在 Qwen3-30B-A3B 上,MOPD 全面超过 Mix-RL、Cascade RL、Off-Policy Finetune 和 Param-Merge 基线,继承每个教师 91–95% 的能力,且多个领域教师可并行独立开发,工程效率显著提升。\n\n落地:MOPD 已被小米 MiMo-V2-Flash 技术报告规模化部署,最近韩国 Upstage 的 Solar Open 2(250B 总参 \u002F 15B 激活)直接把它用在 agent 训练环节;NVIDIA NeMo-RL 在 nightly 版本里把 MOPD 集成为原生算法,advantage estimator 选 opd,异步 GRPO + NeMo Gym 走 agent 多轮交互流水线。\n\n评论:过去很多人把 on-policy distillation 当成「用 teacher logits 训学生」的同义词,MOPD 把它重新定义成「用 teacher 对学生自己 rollouts 上每个 token 的概率差构造 advantage」。这本质上是在做 reverse-KL 最小化,但不需要 teacher 输出完整词表,工程友好很多。对国内做 agent 的团队来说,这意味着可以同时养几个专项教师,再蒸馏出一个统一底座,而不是痛苦地做模型融合。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.30406","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17],{"id":11,"name":12,"slug":12,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[21],{"id":22,"lang":23,"title":24,"summary":25,"content":25},"6257f37e-f250-457a-a76c-e28444445aa9","en","MOPD packs multi-domain RL teachers into one student model","Background: modern large-model post-training almost always relies on reinforcement learning to squeeze out specific capabilities, but stuffing code, math, reasoning, and conversation demands into a single model inevitably sacrifices one for another. Existing Off-Policy Finetune, Mix-RL, Cascade RL, and Param-Merge approaches are either inefficient or lose points in some domain — weights and data are coupled too deeply, making parallel engineering progress hard. Core method: Xiaomi's LLM-Core team, together with Peking University, has published MOPD (Multi-Teacher On-Policy Distillation) in arXiv:2606.30406, taking a different tack — first run RL independently for each domain to get a set of domain teachers, then let the student, on its own generated rollouts, use a token-level stop-gradient distillation advantage Â_t = sg[log π_teacher(t) - log π_student(t)] to update backwards. This single move solves two problems at once: eliminates exposure bias (the student learns on tokens encountered on its own real-distribution rollouts, no longer on out-of-distribution samples from offline datasets); and preserves a dense optimization signal (token-level differences are available immediately, no need to wait for the end-of-episode return). Numerical results: on Qwen3-30B-A3B, MOPD comprehensively beats Mix-RL, Cascade RL, Off-Policy Finetune, and Param-Merge baselines, inheriting 91–95% of each teacher's capability, and multiple domain teachers can be developed in parallel and independently, significantly improving engineering efficiency. Deployment: MOPD has already been deployed at scale in Xiaomi's MiMo-V2-Flash technical report; recently Upstage's Solar Open 2 (250B total \u002F 15B active) in Korea used it directly in agent training; NVIDIA NeMo-RL has integrated MOPD as a native algorithm in its nightly build, with advantage estimator set to opd, running async GRPO + NeMo Gym on agent multi-turn interaction pipelines. Commentary: many people in the past equated on-policy distillation with \"training the student on teacher logits\"; MOPD redefines it as \"using the teacher's per-token probability difference on the student's own rollouts to construct an advantage\". This is essentially reverse-KL minimization, but without needing the teacher to output the full vocabulary — much more engineering-friendly. For Chinese agent teams, this means they can raise several specialized teachers in parallel and then distill a unified base model, rather than painfully doing model merging.","mopd-multi-teacher-on-policy-distillation","2026-07-24T08:00:00Z","2026-07-24T00:07:47.357191Z","2026-08-19T02:08:40.142862Z",true,"agent",312,[34,43],{"slug":35,"tag_slug":35,"title_zh":36,"title_en":37,"intro_zh":38,"intro_en":39,"id":40,"is_active":30,"created_at":41,"modified_at":42},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":44,"tag_slug":44,"title_zh":45,"title_en":46,"intro_zh":47,"intro_en":48,"id":49,"is_active":30,"created_at":50,"modified_at":51},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":53},[54,59,64,69,74,79],{"id":55,"title":56,"news_slug":57,"published_at":58},"b0c4e8d2-5662-4e3e-b489-6202eabbe97b","Dream-RSI 把历史当模拟器:162 倍杠杆重写 RSI 算力账本","dream-rsi-replay-simulator-162x","2026-09-16T06:00:00+00:00",{"id":60,"title":61,"news_slug":62,"published_at":63},"2266cea6-06f1-4932-8905-1bc3f2e5a8c0","Meta FAIR 字节蒸馏研究:End-Of-Token 渐近反超 token 蒸馏 4%,数据只需 1\u002F6","meta-fair-byte-distillation-token-ceiling-2026-09","2026-09-15T02:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"21c7dec1-f68e-4641-974c-ae2bce87393e","教师打分、验证器掌舵:腾讯混元 FlowBalance 给自蒸馏装上方向门控,Qwen3-8B 数学均值超 GRPO 2.12 分","flowbalance-verifier-gated-self-distillation","2026-09-08T15:08:17+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"3d922c00-afcb-4f1c-a6d5-8f9d6c10c642","从 Kimi Linear 到 Kimi K3:MoE 推理效率战里被忽略的架构升级","kimi-k3-latentmoe-kda-attnres-nope","2026-07-30T00:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"b5638cab-a4d6-44ac-9230-32ed0a4cba9d","ARMT 把「记忆」焊进 Transformer:用恒定显存换无限上下文","armt-associative-recurrent-memory-transformer","2026-07-23T00:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"d2844cbb-b70e-469c-95b8-71cee8d735a6","给 Transformer 装上「CNN 鼻子」:用 0.01% 的参数量换 benchmark 普涨","transformer-cnn-nose-0-01-percent","2026-07-22T12:10:00+00:00"]