[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-4d92e0b1-04a3-4524-9ae3-b8456aa74f2a":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"4d92e0b1-04a3-4524-9ae3-b8456aa74f2a","NAVER 提出 On-Policy Delta Distillation:用「差分信号」重新定义推理蒸馏","NAVER AI Lab 于 2026 年 7 月 16 日在 arXiv 发布《On-Policy Delta Distillation》(arXiv:2607.15161),为推理大模型的后训练蒸馏提供了一条更直接、更经济的路径,作者为 Byeongho Heo、Jaehui Hwang、Sangdoo Yun 与 Dongyoon Han。\n\n传统 on-policy distillation 的目标,通常直接让学生模仿教师的输出分布。然而这一目标不可避免地混入了教师自身的语言先验——学生既要学推理,又要承担教师对世界的所有已学先验,代价偏高,效率受限。NAVER 团队的核心洞察是:推理能力的本质,源自「教师相对其同源 base 模型的增量变化」。论文把这一增量定义为 delta signal,即「教师模型与其同源 base 模型在每个 token 上的分布差」,精准剥离预训练残留,只留下「指令微调或推理 RL 过程中新引入的成分」。\n\n基于 delta signal 重新设计的 OPD² 蒸馏目标,在数学、科学、代码推理基准上一致优于传统 on-policy distillation,学生模型仅需极短的后训练周期即可逼近教师表现。论文报告的实验覆盖 19 页正文、4 张图、12 张表,具有相对完整的实证支撑。代码将于 github.com\u002Fnaver-ai\u002Fopd2 开源,后续可接入到主流 RL 后训练流程中。\n\n这条路径更深层的意义在于,把「蒸馏什么」从「模仿整体输出分布」重新校准为「迁移教师在 RL\u002F指令微调中真正学到的新能力」。它延续了过去半年学界对 reasoning post-training 目标函数的精细化讨论——从 token 级 imitation 到 preference-based RL,再到 delta-based distillation,目标粒度越做越细。对追求小模型继承大模型推理能力、又要控制训练成本的研究与工程团队,这是近几个月里少有的、机制层面而非工程层面真正推进的进展。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.15161","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer","2026-07-18T16:07:00Z","2026-07-18T16:13:58.387601Z","2026-07-18T16:13:58.387613Z",true,"agent",5]