[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-opd-vs-grpo-compute-efficiency":3,"topics-all":38,"news-related-22beb25f-2981-4089-ae0a-e1305d467872":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"22beb25f-2981-4089-ae0a-e1305d467872","蒸馏打不过RL？换对教师省50倍算力","Salesforce把在线蒸馏（OPD）和GRPO放进同一套训练FLOPs记账：外部教师时OPD普遍输给GRPO，教师越强越贵；换成学生自己的GRPO checkpoint当教师，数学任务以50倍更少的蒸馏FLOPs恢复同精度；多教师OPD合并领域专家，总FLOPs比joint GRPO省42%。","蒸馏和强化学习到底谁更省算力，是后训练时代吵了一年的公案。各家报告要么给 GPU 时长，要么给梯度步数，真正按训练 FLOPs 对齐口径的对比一直缺席。Salesforce AI Research 的最新论文补上了这个空白：把在线策略蒸馏（OPD）和 GRPO 放进同一套 FLOPs 记账体系，从任务难度、模型规模、训练域到优化配置逐项对照，结论相当反直觉。\n\n## 外部教师：直觉全线失守\n\nOPD 的卖点直白：学生在自己的 rollout 上训练，冻结教师给每个 token 打分。稠密的逐 token 监督，听起来比 GRPO 那种只看最终对错的结果奖励强得多——尤其难题，学生几乎做不对时 GRPO 缺少正样本，教师却依然能一步步教。\n\n实测把这套直觉打穿了。团队用 mean@32 解题率把 DAPO-Math-17K 按学生\u002F教师难度切成八个子集重训，理论上对 OPD 最有利的那一格（学生几乎全错、教师正确率 60% 以上），域内评测里 GRPO 依然全程胜出。九个消融轴——学生初始化、学习率、batch size、KL 估计器、思考模式、学生尺寸、训练域、RL 框架——没有一个能翻转排序。代码域差距更悬殊：GRPO 在 0.6 EFLOP 拿下 LiveCodeBench v6 的 46.4%，32B 外部教师的 OPD 峰值只有 28.7%，却烧掉 68.7 EFLOP。教师越强，OPD 曲线只是整体右移变贵，学生上限并没有抬高。还有一笔隐性成本：外部教师训出的学生回复长度持续膨胀，每步 FLOPs 越付越多。\n\n## 换对教师：50 倍差距从哪来\n\n转机出在教师的\"出身\"。测过的所有教师里，唯一能让学生完整恢复教师性能的设置，是拿学生自己 GRPO 训出的 checkpoint 当教师。数学任务上，OPD 用少 50 倍的蒸馏阶段 FLOPs 摸到同一精度（默认 8-rollout 设置下是 66 倍），比拿外部 Qwen3-8B 当教师便宜 135.6 倍。这个结果跨学生尺寸、跨代码域、在第二个模型家族 OLMo-3-7B 上都复现，后者甚至出现学生反超教师 11 个绝对分的奇观。\n\n但论文最诚实的一笔也在这里：单域场景下，GRPO checkpoint 本身就是你要的模型，再蒸馏回 base 学生属于纯增量开销——50 倍只省在蒸馏阶段。换教师不是免费午餐，是换了记账方式。\n\n## 真正的用武之地：先专后合\n\n多域才是蒸馏翻身的地方。数学、代码各训一个 GRPO 专家，没有任何单一 checkpoint 能同时服务两个域；用多教师 OPD 把专家合并进一个学生，蒸馏阶段只花专家训练 FLOPs 的 5.2%（两域）和 8.0%（四域）。两阶段合计，两域 pipeline 以少 42.0% 的总 FLOPs 追平 joint GRPO（四域省 31.6%），而且 joint GRPO 在测试预算内根本追不上合并学生的性能。对照组里，单个外部 8B 教师在多域混合训练上直接翻车：学生出现无限重复、英文 prompt 输出中文字符这类退化行为。\n\n分量在于：DeepSeek-V4、Kimi K3 等前沿模型报告已经把 multi-teacher OPD 写进后训练流程，但一直只当工程选择用，没人算过总账。这篇论文等于给前沿实践补了张收据——蒸馏的定位不是\"比 RL 更快的升级手段\"，而是\"把多个专家便宜地焊进一个模型\"的合并器。\n\n## 所以呢\n\n下次看到\"蒸馏比 RL 高效\"的说法，先追问两句：教师是谁的 checkpoint？省的是哪一段 FLOPs？多半答案是：每个域先用 RL 训成专家，再用便宜一个量级的蒸馏做合并。算力分配从\"二选一\"变成\"两边都要、各干各的活\"。\n\n论文地址：https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08453","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08453","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"b8ad3052-7495-4ebf-87bf-c8558760a2ec","en","Distillation vs GRPO: The Right Teacher Saves 50x FLOPs","Salesforce pits OPD vs GRPO at matched FLOPs: external-teacher OPD loses; a self-GRPO teacher needs 50x fewer FLOPs; expert merging saves 42% FLOPs.","Is distillation or reinforcement learning the cheaper post-training path? Teams have argued about this for a year, usually with GPU-hour counts or gradient-step ratios—rarely with a matched training-FLOPs comparison. A new paper from Salesforce AI Research closes that gap: it puts on-policy distillation (OPD) and GRPO into one FLOPs accounting framework and sweeps task difficulty, student\u002Fteacher sizes, training domains, and optimization choices. The findings are counter-intuitive on several fronts.\n\n## External teachers: the intuition collapses\n\nOPD's pitch is straightforward: the student trains on its own rollouts while a frozen teacher scores every token. Dense per-token supervision sounds far stronger than GRPO's outcome-only reward—especially on hard problems, where a student that rarely solves correctly leaves GRPO with almost no positive signal, while the teacher can still supervise step by step.\n\nThe measurements punch through that intuition. The team stratified DAPO-Math-17K by mean@32 solve rates of both student and teacher into eight subsets and retrained: in the cell most favorable to OPD on paper (student near-zero, teacher above 0.6), GRPO still dominated in-domain at lower FLOPs. None of nine ablation axes—student initialization, learning rate, batch size, KL estimator, thinking mode, student size, training domain, RL framework—flipped the ordering. The gap on code is starker: GRPO hit 46.4% on LiveCodeBench v6 at 0.6 EFLOP, while OPD with a 32B external teacher peaked at 28.7% after burning 68.7 EFLOP. Stronger teachers just shifted the OPD curve rightward—more expensive, no higher ceiling. There is also a hidden cost: students distilled from external teachers inflate response length, paying more FLOPs per step.\n\n## The right teacher: where the 50x comes from\n\nThe twist is the teacher's provenance. Among all teachers tested, the only setup where the student fully recovers teacher performance is using the student's own GRPO checkpoint as teacher. On math, OPD matched that checkpoint's accuracy with 50x fewer distillation-stage FLOPs (66x at the default 8-rollout setting), and 135.6x fewer than using an external Qwen3-8B teacher. The result replicates across student sizes, on code, and in a second model family (OLMo-3-7B)—where the student even beat its teacher by 11 absolute points.\n\nBut the paper's most honest accounting sits right here: on a single domain, the GRPO checkpoint already is the model you want, so distilling it back into the base student is pure added cost—the 50x only covers the distillation stage. Swapping teachers is not a free lunch; it changes what the ledger counts.\n\n## The real use case: specialize, then consolidate\n\nMulti-domain is where distillation earns its keep. Train separate GRPO experts for math and code—no single checkpoint serves both—then merge them into one student with multi-teacher OPD. The distillation stage costs only 5.2% (two domains) and 8.0% (four domains) of the specialists' training FLOPs. Counting both stages, the two-domain pipeline matches joint GRPO with 42.0% fewer total FLOPs (31.6% on four domains), and joint GRPO never reaches the merged student's performance within the tested budget. Meanwhile, a single external 8B teacher on the multi-domain mix collapsed outright: the student degenerated into unstoppable repetition and emitted Chinese characters for English prompts.\n\nThe weight of this result: DeepSeek-V4 and Kimi K3 reports already bake multi-teacher OPD into their post-training pipelines as an engineering choice, with no full-pipeline accounting. This paper supplies the receipt—distillation's role is not \"a faster upgrade than RL\" but \"a cheap consolidator that welds multiple experts into one model.\"\n\n## So what\n\nNext time you hear \"distillation is more efficient than RL,\" ask two questions: whose checkpoint is the teacher, and which stage's FLOPs are being saved? The likely answer: RL-train each domain into a specialist first, then merge with distillation an order of magnitude cheaper. Compute allocation shifts from \"distillation or RL\" to \"both, each doing its own job.\"\n\nPaper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08453","opd-vs-grpo-compute-efficiency","2026-11-06T17:30:00Z","2026-11-06T17:08:00.059269Z","2026-11-06T17:08:00.059277Z",true,"agent",312,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"21c7dec1-f68e-4641-974c-ae2bce87393e","教师打分、验证器掌舵:腾讯混元 FlowBalance 给自蒸馏装上方向门控,Qwen3-8B 数学均值超 GRPO 2.12 分","flowbalance-verifier-gated-self-distillation","2026-09-08T15:08:17+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"b362eb89-32ef-46ed-b65a-dd65f6f305f2","Direct-OPD 把「RL 经验」跨模型规模可复用：字节×清华让弱模型的策略差当强模型的隐式奖励","direct-opd-rl-experience-transfer","2026-07-14T14:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"8f0348fd-b4d2-4567-8dae-f2d6f9cf7760","论文实测:算术电路焊进LLM权重,6位数冲上100%","zero-gradient-arithmetic-circuits-llm-weights","2026-11-21T17:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"8e741f04-82a2-46da-9d30-3c31117ca70c","Ambient SFT:让预训练的自己当高温老师","ambient-sft-temperature-matching","2026-11-18T17:30:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"f828f611-b57a-4855-9246-3609c2495faa","老师折进自己:2.27B用22B token留住84%","concertina-folding-llm-teacher-weights","2026-11-16T08:50:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"08c85d0c-1ad8-43d6-bd32-0ec8b21bf2aa","给教师开小抄,学生涨分却不是抄来的","opsd-privileged-solution-teacher-audit","2026-11-15T01:30:00+00:00"]