[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-2026-distillation-three-paradigms":3,"topics-all":36,"news-related-054e060c-e182-42a9-b3ed-229feb8ac0ac":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"054e060c-e182-42a9-b3ed-229feb8ac0ac","2026 年的蒸馏长什么样:Hugging Face 拆解前沿模型三大范式","Hugging Face 工程师 Sergio Paniego 把过去一年各家前沿实验室的蒸馏用法梳理成三条主线。\n\n**第一类:经典「大老师带小学生」**。Gemma 3\u002F4、DeepSeek-R1-Distill 走这条路,把大 teacher 在 next-token 分布或生成文本上的能力压到小尺寸 student。\n\n**第二类:用蒸馏把多个 RL 专家合并成一个学生**——这是今年各家真正收敛的方向。DeepSeek-V4 在数学、代码、Agent 各训一个领域 expert,再 on-policy distillation 合并回单一模型;MiMo-V2-Flash 命名 MOPD;NVIDIA Nemotron 3 Ultra 推到十多个 teacher;GLM-5 用它找回 RL 后期遗忘的能力。Qwen3 给出关键数字:这条路 GPU 小时只有纯 RL 的 1\u002F10,效果反而更好。\n\n**第三类:self-distillation**。Cursor Composer 2.5 用「带 hint 的自己」对齐「不带 hint 的自己」;Thinking Machines 用「上一版自己」蒸馏回「这一版自己」,直接解持续学习难题。\n\n三条线本质是同一个 teacher-student 在不同尺度的回归——蒸馏正从「压缩工具」演变为「训练范式」。","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fsergiopaniego\u002Fdistillation-2026","24d5c6c5-6573-4180-a1fd-f1459842d1af",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"13f63f43-e282-4ee7-ba00-025eebe6c550","en","Hugging Face dissects 2026's three distillation paradigms","Hugging Face engineer Sergio Paniego sorts the past year's distillation usage at the frontier labs into three main lines. **The first: the classic \"big teacher with small student\"**. Gemma 3\u002F4 and DeepSeek-R1-Distill take this path, compressing the large teacher's capability on the next-token distribution or generated text down to a small-sized student. **The second: using distillation to merge multiple RL experts into one student** — this is the direction that everyone has truly converged on this year. DeepSeek-V4 trains domain experts for math, code, and Agent separately, then merges them back into a single model via on-policy distillation; MiMo-V2-Flash names it MOPD; NVIDIA Nemotron 3 Ultra scales up to a dozen teachers; GLM-5 uses it to recover capabilities forgotten late in RL. Qwen3 gives the key number: this path costs only 1\u002F10 of pure RL GPU-hours, and the effect is actually better. **The third: self-distillation**. Cursor Composer 2.5 uses \"itself with hints\" to align with \"itself without hints\"; Thinking Machines uses \"previous-version self\" to distill back to \"current-version self\", directly solving the continual-learning problem. The three lines are essentially the same teacher-student regression at different scales — distillation is evolving from a \"compression tool\" to a \"training paradigm\".","2026-distillation-three-paradigms","2026-07-09T12:00:00Z","2026-07-09T12:07:00.315129Z","2026-08-19T02:08:40.142862Z",true,"agent",222,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"ab2d6e9e-8890-4ae7-b6ca-8febc831a279","HPLT MultiSynt\u002FMT：4.8 万亿 token 多语种数据集","hplt-multisynt-multilingual-dataset","2026-07-07T16:01:00+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"93dfc6f4-e4a9-47a3-aa66-b4b9cee864e7","Beyond LoRA 不只是口号：HF 给 40+ PEFT 方法拍下公平基准，OFT 在图像任务上反超 LoRA","beyond-lora-hf-peft-benchmark-oft","2026-06-18T12:00:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"28c41f06-d20f-481c-b133-cd109af3aed1","答对之后停不下来:微软团队揪出在线蒸馏的 EOS 错配元凶","eos-mismatch-opd-length-inflation","2026-09-18T21:09:06+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"8def771a-d936-4859-930d-02c3011dc55c","LimiX-2 开源：一个模型吃下分类回归插补，表格三榜登顶","limix-2-tabular-foundation-model","2026-09-17T21:09:27+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"176b4807-da61-479f-a514-9381cd13319e","SP3O:3 个锚点修复 PPO critic 的平坦化","sp3o-sparse-critic-supervision","2026-09-17T17:10:01+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"7bae3d71-a5c2-4588-95e7-b5d4b5c7085a","开源模型 4.4 个月追上闭源前沿:Hugging Face 被 NVIDIA 129 亿美元收编","nvidia-acquires-hugging-face-open-source-ai","2026-09-17T08:00:00+00:00"]