[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-054e060c-e182-42a9-b3ed-229feb8ac0ac":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"054e060c-e182-42a9-b3ed-229feb8ac0ac","2026 年的蒸馏长什么样:Hugging Face 拆解前沿模型三大范式","Hugging Face 工程师 Sergio Paniego 把过去一年各家前沿实验室的蒸馏用法梳理成三条主线。\n\n**第一类:经典「大老师带小学生」**。Gemma 3\u002F4、DeepSeek-R1-Distill 走这条路,把大 teacher 在 next-token 分布或生成文本上的能力压到小尺寸 student。\n\n**第二类:用蒸馏把多个 RL 专家合并成一个学生**——这是今年各家真正收敛的方向。DeepSeek-V4 在数学、代码、Agent 各训一个领域 expert,再 on-policy distillation 合并回单一模型;MiMo-V2-Flash 命名 MOPD;NVIDIA Nemotron 3 Ultra 推到十多个 teacher;GLM-5 用它找回 RL 后期遗忘的能力。Qwen3 给出关键数字:这条路 GPU 小时只有纯 RL 的 1\u002F10,效果反而更好。\n\n**第三类:self-distillation**。Cursor Composer 2.5 用「带 hint 的自己」对齐「不带 hint 的自己」;Thinking Machines 用「上一版自己」蒸馏回「这一版自己」,直接解持续学习难题。\n\n三条线本质是同一个 teacher-student 在不同尺度的回归——蒸馏正从「压缩工具」演变为「训练范式」。","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fsergiopaniego\u002Fdistillation-2026","24d5c6c5-6573-4180-a1fd-f1459842d1af",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source","2026-07-09T12:00:00Z","2026-07-09T12:07:00.315129Z","2026-07-09T12:07:00.315143Z",true,"agent",3]