[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-waypoint-opd-direction-gated-distillation":3,"topics-all":38,"news-related-d1b7ef56-357c-4b38-841e-4d4d60970490":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"d1b7ef56-357c-4b38-841e-4d4d60970490","Waypoint-OPD给蒸馏加方向门控：教师前后打架就不学","on-policy 蒸馏只看教师 RL 前后的终点差，但同样的位移可能来自方向一致的强化，也可能是中途互相抵消的摆动。Waypoint-OPD 用中间 checkpoint 拆出早晚期位移，方向一致才放行、相反就抑制，只多一次冻结教师打分；作者称其在 Qwen3 六组数学与代码设置中均拿到对比方法中的最高域平均分。","开源圈这两年最热的省钱手艺，是把别人花大钱 RL 训出来的推理能力，用 on-policy 蒸馏搬进自己的模型。但 10 月 15 日出现在 arXiv 的一篇论文指出一个被忽略的坑：蒸馏时只看教师 RL 前后的「终点差」，可能把不该学的东西也学进来。这篇 [Waypoint-OPD](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.21601) 已经冲上 Hugging Face 日榜第一，拿到 665 个赞。\n\n## 终点差会骗人\n\n先补一句背景。所谓 policy-shift 蒸馏，是把教师模型强化学习前后的 log 概率之差当作监督信号，用在学生自己生成的轨迹上。问题在于，这个「前后之差」是个端点快照：它说不清教师内部到底发生了什么。论文的核心观察是——同样的终点差，可能来自全程方向一致的稳定强化，也可能来自先往东、再往西、中途互相抵消的两次摆动。对学生来说这两种情况长得一模一样，但学习价值完全不同。\n\n## 加一道方向门控\n\nWaypoint-OPD 的解法很直接：从同一次 RL 训练里取一个中间 checkpoint 当「路标」，把总位移拆成早期位移和晚期位移，再在学生实际访问到的前缀上比较两段的方向。方向一致，说明教师的偏好演化是连贯的，放行终点位移；方向相反，说明这个位置的位移大概率是噪声或中间态，直接抑制。代价方面，论文称相比只传终点差，整个方法只多了一次冻结教师的打分。\n\n## 实验与复现\n\n实验配置：教师是 RL 训练过的 Qwen3-4B，学生是 Qwen3-8B、Qwen3-14B 和 Qwen3-30B-A3B，任务覆盖数学与代码推理。论文报告称，在全部六个「领域×学生」设置中，该方法都拿到了所对比蒸馏方法里的最高域平均分。消融实验里，把门控去掉或者随机打乱，数学推理成绩都会回落。复现层面，作者在 GitHub 放出了基于 verl 的实现（仓库 [hhhbxloveai\u002FWaypointOPD](https:\u002F\u002Fgithub.com\u002Fhhhbxloveai\u002FWaypointOPD)，以匿名实现名义发布），默认单机 8 卡，训练集用 DAPO-17K，评估用 AIME24，路标教师对应 Step600、最终教师对应 Step1200 两个 checkpoint——模型权重本身不含在仓库里。\n\n## 真正有意思的点\n\n一个容易被忽略的细节：教师只有 4B，学生反而是 8B 到 30B。这不是传统的「大教小」，而是把小模型 RL 出来的增益迁移给更大的底座——蒸馏的对象不是知识，而是强化学习带来的策略改进。另一个信号是，蒸馏研究正在从「学什么」转向「什么时候不学」：门控本质上是在给监督信号降噪，只在教师自身演化一致的位置才下注。对开源社区来说成本账也很清楚——RL 训练贵，但中间 checkpoint 是现成副产品，多一次打分开销几乎可以忽略。\n\n所以下次跑 RL，别只盯着最终 checkpoint：中间那个「路标」，可能才是蒸馏时最值钱的资产。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.21601","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"5a2326c5-c450-4972-8feb-9b82a7d2b5c6","en","Waypoint-OPD: Direction-Gated On-Policy Distillation","Waypoint-OPD gates on-policy distillation by agreement between early and late teacher shifts, needing only one extra frozen-teacher scoring pass.","Open-weight teams keep costs down by distilling the reasoning ability that others paid fortunes to RL-train. But a paper posted to arXiv on October 15 flags an overlooked trap: supervising distillation with only the teacher's pre\u002Fpost-RL endpoint shift can import exactly the wrong signal. The paper, [Waypoint-OPD](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.21601), has since topped the Hugging Face daily paper chart with 665 upvotes.\n\n## Endpoint shifts can lie\n\nQuick background. Policy-shift distillation takes the difference between a teacher's log probabilities before and after reinforcement learning and applies it as supervision on trajectories the student itself generates. The problem is that this difference is an endpoint snapshot: it cannot say what actually happened inside the run. The paper's core observation is that identical endpoint shifts can arise from a consistent, direction-stable strengthening — or from an early swing one way and a late swing back that partially cancel each other. To a student these two cases look exactly the same, yet their learning value is completely different.\n\n## A directional gate\n\nThe fix is direct: pull an intermediate checkpoint from the same RL run as a waypoint, split the total shift into an early and a late component, then compare their directions on prefixes the student actually visits. When the two directions agree, the teacher's preference evolution was coherent, and the endpoint shift passes through as supervision. When they oppose, the shift at that position is likely noise or a transient intermediate state, and it gets suppressed. Cost-wise, the authors report that the whole mechanism adds only one extra frozen-teacher scoring pass compared with endpoint-only transfer — a nearly free price for the filtering effect.\n\n## Experiments and reproduction\n\nThe setup: teachers are RL-trained Qwen3-4B models; students are Qwen3-8B, Qwen3-14B, and Qwen3-30B-A3B; tasks span mathematical and code reasoning. The authors report that across all six domain-student settings, the method achieves the highest domain-average score among the compared distillation methods. Ablations with the Qwen3-8B student show that removing or shuffling the gates reduces mathematical reasoning performance, supporting the claim that directional agreement — not just any gating — carries the effect. For reproduction, the authors released a verl-based implementation on GitHub ([hhhbxloveai\u002FWaypointOPD](https:\u002F\u002Fgithub.com\u002Fhhhbxloveai\u002FWaypointOPD), published as an anonymous implementation), with a default single-node 8-GPU configuration, DAPO-17K as the training set, AIME24 for evaluation, and the waypoint and final teachers mapped to the Step600 and Step1200 checkpoints. Model weights are not included in the repository.\n\n## Why it matters\n\nOne easily missed detail: the teacher is only 4B while the students run from 8B to 30B. This is not the classic big-teaches-small setup — what gets transferred is the gain that RL produced, grafted onto larger bases. The object of distillation is not knowledge but the policy improvement reinforcement learning induced. A second signal: distillation research is visibly shifting from asking what to learn to asking when not to learn. The gate is essentially denoising the supervision signal, betting only where the teacher's own evolution was self-consistent. For the open-source community the cost arithmetic is equally clear — RL runs are expensive, but intermediate checkpoints are a free byproduct of every run, and one extra scoring pass is negligible.\n\nSo the next time you finish an RL run, don't just keep the final checkpoint. That intermediate waypoint may be the most valuable asset you own for distillation.","waypoint-opd-direction-gated-distillation","2026-10-17T23:20:00Z","2026-10-17T23:07:15.113211Z","2026-10-17T23:07:15.113227Z",true,"agent",134,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"3a419e1c-fb2b-467e-a5b6-a7188525bc03","TraceForge:推理痕迹自己造,学生反超教师真迹","traceforge-family-native-distillation","2026-10-22T15:10:42+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"d941056b-c2e7-42e5-965a-a982c20b1169","Qwen3.8-Flash-Next 架构细节:Gated Residual 多分支残差 + QSA micro-block 稀疏注意力","qwen3-8-flash-next-cost-efficiency-architecture","2026-09-02T02:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"fd29c2f1-175f-416e-bde2-95dd65ca3465","KronLSTM 精准改写记忆:线性注意力长上下文新解","kronlstm-kronecker-delta-rule","2026-10-20T15:06:16+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"559df680-f13e-48f0-9263-73116aaff411","上交大DLite免历史投机解码:Agent提速2.74倍","dlite-history-free-speculative-decoding","2026-10-19T17:07:52+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"48726cf0-6f0c-4b09-841e-06aa8b0ffcfd","上海AI Lab Intern-Decision:多模态决策模型,4B 微超 Jev 90.02%","intern-decision-qwen3-5-multimodal-decision-model","2026-10-18T11:13:06+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"eeb3f139-8df8-40d7-a1e6-82e49b668547","VideoJev:视频问答提速100倍,不生成只打分","videojev-system-one-video-understanding","2026-10-14T15:09:17+00:00"]