[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-route-dmd-speech-distillation-separation-collapse":3,"topics-all":38,"news-related-88a727c6-311a-4cf4-b5c3-98d4251d956a":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"88a727c6-311a-4cf4-b5c3-98d4251d956a","语音蒸馏踩雷:分离任务从+6.6掉到-1.7","Route-DMD 论文:分布匹配蒸馏用在统一语音基础模型上会崩掉分离能力,不加路由时 SI-SDRi 从 +6.6dB 掉到 -1.7dB,PER 从 7.1% 升到 54.4%。根因不在学生而在教师——教师收到偏离分布的加噪状态后预测不可靠。修法是分离改用回归损失,4 步学生提速 4.5 倍。","语音基础模型正在把生成、编辑、分离、增强塞进同一个网络，一条自然语言指令走到底。听起来很美，但少步蒸馏一进场，账就不好算了：分布匹配蒸馏（DMD）在生成任务上跑得好好的，换到分离任务上，能把学生模型练到比原始混音还差。\n\nRoute-DMD 这篇论文（[arXiv:2611.08011](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08011)）把这个失效过程完整拆开了。动手的是上海交大、BIGAI、上海创新研究院、南洋理工和腾讯混元的联合团队，实验用的教师模型正是今年 9 月开源的那个统一语音基础模型 AuK（[arXiv:2609.08936](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.08936)）。\n\n## 摘掉生成任务，分离任务就崩了\n\n先把成本摆出来。语音基础模型走流匹配，每个请求要 20–50 次迭代步，而分类器自由引导（CFG）会把每一步的计算量再翻一倍。少步蒸馏就是冲着这个来的——用一个几步出结果的\"学生\"换掉多步的\"教师\"。\n\n论文指出，此前的语音蒸馏工作全都只在生成任务上验证过，而一个同时支持编辑和语音处理的基础模型该怎么蒸馏，\"仍然无人探索\"。Route-DMD 想把这条路走通，第一步却是先把它走坏：直接把 DMD 套上去，分离能力立刻崩盘。\n\n消融实验给出了崩坏的样子：不加路由时，分离任务的 SI-SDRi（尺度无关信号失真比改善量）从 +6.6 dB 一路掉到 -1.7 dB，音素错误率 PER 从 7.1% 涨到 54.4%，UTMOS 和说话人相似度同步跳水。\n\n## 崩的原因不在学生，在教师\n\n论文给出的机制解释挺有意思。分布匹配的监督信号来自教师在\"加噪状态\"上的打分——但教师只在训练分布附近可靠。一旦学生早期输出是错的，这份错误输出加噪之后的状态就离教师的训练分布很远，教师在这个位置给出的\"真实分数\"本身就不可靠。用一个不可靠的信号去监督模型，训得越狠，崩得越狠。\n\n有个对照实验把病灶钉死了：把喂给教师的加噪对象换成加噪的真值目标（而不是学生的错误输出），DMD 照常在这些样本上跑，SI-SDRi 稳在 +5.1 到 +5.4 dB，PER 控制在 7.0%–8.4%，崩溃没有发生。论文的结论是：问题出在教师收到的输入上。\n\n## 修法：把\"匹配分布\"换成\"回归真值\"\n\n既然病因清楚了，修法就顺理成章。论文的核心洞察是：分布匹配这个目标不是万能的。生成任务给的是一句话描述，答案是一个分布；但分离任务在指令给定后只有唯一正确答案——把谁从混音里挑出来，是有标准答案的。对这种任务，回归损失比分布匹配更合适。\n\n于是 Route-DMD 按任务路由：说话人分离和人声分离这两类被摘出来，改用干净端点上的回归损失直接拟合真值潜变量，其余任务保留解耦分布匹配。两级训练：先做 500 次更新的一致性蒸馏初始化（教师 CFG 引导尺度 2），再进 Route-DMD 阶段。\n\n另一处替换是 CFG 换成 APG（自适应投影引导）。高引导尺度下 CFG 会沿条件方向过度外推产生伪影，而这些伪影会被学生一并学走。APG 在干净预测空间里分解条件-无条件残差，保留正交分量、按系数 η 削弱平行分量。消融里对比很清楚：用 CFG 学生前期涨得快但很快饱和，用 APG 则能继续爬，最终 SIM 更高、WER 更低。\n\n## 表格里的两处回退，比摘要诚实\n\n摘要说 4 步学生\"追平\"32 步教师，4.5 倍端到端加速。加速是真的：单张 NVIDIA H20 上端到端实时率 RTF 从 0.4333 降到 0.0958，算出来 4.52 倍，和 4.5 倍对得上；主观听测的自然度 N-MOS 从 4.23±0.10 升到 4.34±0.10，指令遵循 I-MOS 从 4.50±0.09 升到 4.57±0.09，论文说两者都在报告的 95% 置信区间内与教师持平。\n\n但表格里有回退，得自己看。中文指令 TTS 的 DSD-ZH 准确率，AuK 32 步是 83.4，Route-DMD 4 步是 78.8，掉了 4.6 分，甚至低于对照的 Qwen3-TTS-VD 的 81.1。语音编辑 MMAE-Speech 上，指令遵循率 IFR 从 48.23 降到 46.62，一致率 CR 从 88.11 降到 86.41，只有精确匹配率 EMR 从 12.44 升到 13.85。SeedTTS-Eval 平均 WER 也从 2.65 升到 2.85。分离任务上 Libri2Mix 的 WER 同样从 9.12 升到 10.07。\n\n另一头的好处也很实在：三个语音处理基准的 OVRL 和 UTMOS 全面超过教师（CHiME-4 3.28→3.35、Libri2Mix 3.28→3.32、VCTK-SR 3.21→3.25），中文难子集 test-zh-hard 拿下最好的 SIM 0.784。\n\n## 所以呢\n\n这篇论文的价值可能不在 4.5 倍，而在它示范了一个容易被跳过的判断：多任务统一模型到了蒸馏阶段，不能一个损失打天下。\n\n训练侧 AuK 其实已经按任务分化过后训练——开放式编辑走人类反馈偏好优化，语音生成走基于奖励的强化学习。轮到蒸馏，问题在于教师要充当\"评分器\"，而不同任务对评分器的可靠性要求完全不同。分布匹配要求教师在学生可达的样本空间里给出可信分数，一旦某个任务天然有唯一解、错误输出又离训练分布很远，这个前提就破了。\n\n更实用的一条是读表别读摘要。\"4 步追平 32 步\"在中文指令 TTS 上差了 4.6 分，在编辑的 IFR\u002FCR 上也各有回落，收益集中在处理类任务和极难子集上。这对做语音 Agent 的团队很实际：如果你打算把统一语音模型蒸馏成少步版本，先想清楚自己要保住的是生成质量、指令跟随，还是处理能力——三者不是同时保住的。\n\n论文与代码：[arXiv:2611.08011](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08011) · [项目主页](https:\u002F\u002Froute-dmd.github.io\u002F) · [GitHub 仓库](https:\u002F\u002Fgithub.com\u002FiHateTheWorld555\u002FAuK-RouteDMD)\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08011","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"64ddd0ec-c37a-4d8e-bccc-b8b6b1409c3a","en","Route-DMD: Speech Distillation Collapses on Separation","Route-DMD: distribution matching distillation collapses speech separation. Without routing, SI-SDRi falls from +6.6 dB to -1.7 dB and PER hits 54.4%.","Speech foundation models are being consolidated into a single network that handles generation, editing, separation and enhancement behind one natural-language instruction. That sounds elegant, but few-step distillation turns the accounting ugly: distribution matching distillation (DMD) works fine on generation, and when you move it to separation, it can train the student model to be worse than the original mixture.\n\nThe Route-DMD paper ([arXiv:2611.08011](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08011)) takes this failure apart in full. The work comes from a joint team across Shanghai Jiao Tong University, BIGAI, the Shanghai Innovation Institute, Nanyang Technological University and Tencent Hunyuan, and the teacher model used in the experiments is AuK, the unified speech foundation model open-sourced back in September ([arXiv:2609.08936](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.08936)).\n\n## Pull generation away, and separation collapses\n\nStart with the cost. Speech foundation models built on flow matching need 20–50 iterative steps per request, and classifier-free guidance (CFG) doubles the computation at every one of those steps. Few-step distillation targets exactly this: swap a multi-step teacher for a \"student\" that produces a result in a handful of steps.\n\nThe paper notes that prior speech distillation work was validated solely on generation tasks, and that distilling a speech foundation model which also supports editing and speech processing \"remains unexplored\". Route-DMD set out to open that road, and its first step was to break it: applying DMD directly destroys separation capability.\n\nThe ablation shows what the damage looks like. Without routing, the separation task's SI-SDRi (scale-invariant signal-to-distortion ratio improvement) falls from +6.6 dB all the way to -1.7 dB, while phone error rate (PER) climbs from 7.1% to 54.4%, with UTMOS and speaker similarity dropping in step.\n\n## The fault lies with the teacher, not the student\n\nThe mechanism the paper proposes is worth dwelling on. The supervision signal for distribution matching comes from the teacher scoring a noised state — but the teacher is only reliable near its training distribution. Once the student's early outputs are wrong, the noised version of those wrong outputs sits far away from where the teacher was trained, so the \"real score\" the teacher returns there is itself unreliable. Supervise a model with an unreliable signal and the more you train, the harder it collapses.\n\nOne control experiment nails the pathology down. Feed the teacher a noised ground-truth target instead of the noised student output, and DMD still runs on those samples without the collapse: SI-SDRi holds at +5.1 to +5.4 dB and PER stays within 7.0%–8.4%. The paper's conclusion is that the failure is located in the input the teacher receives.\n\n## The fix: swap \"match the distribution\" for \"regress to the truth\"\n\nWith the diagnosis in hand, the fix follows. The core insight is that a distribution-matching objective is not universal. Generation is described by a sentence and its answer is a distribution; separation, once the instruction is given, has a single correct answer — pulling a particular voice out of a mix has a ground truth. For that kind of task, a regression loss fits better than distribution matching.\n\nSo Route-DMD routes by task. Speaker separation and vocal separation are pulled out and trained with a clean-endpoint regression loss that fits the ground-truth latent directly; every other task keeps decoupled distribution matching. Training runs in two stages: first a consistency distillation initialization of 500 updates (teacher CFG at guidance scale 2), then the Route-DMD stage itself.\n\nA second substitution replaces CFG with APG (adaptive projected guidance). At high guidance scales CFG over-extrapolates along the conditional direction and produces artifacts, and the student absorbs those artifacts along with everything else. APG operates in clean-prediction space, decomposing the conditional-minus-unconditional residual with respect to the conditional prediction, keeping the orthogonal component and down-weighting the parallel one by a coefficient η. The ablation is unambiguous: with CFG the student improves faster at first but saturates quickly, while with APG it keeps climbing and finishes with higher SIM and lower WER in both languages.\n\n## Two regressions the table shows and the abstract does not\n\nThe abstract says the 4-step student \"matches\" the 32-step teacher with a 4.5× end-to-end speedup. The speedup is real: on a single NVIDIA H20 the end-to-end real-time factor (RTF) drops from 0.4333 to 0.0958, which works out to 4.52× and matches the claimed 4.5×. Subjective evaluation moves the right way too, with naturalness MOS going from 4.23 ± 0.10 to 4.34 ± 0.10 and instruction adherence from 4.50 ± 0.09 to 4.57 ± 0.09 — the paper says both sit within the reported 95% confidence intervals of the teacher.\n\nBut there are regressions in the table, and you have to read them yourself. On Chinese instruction-following TTS (InstructTTSEval DSD-ZH), the 32-step AuK scores 83.4 while the 4-step Route-DMD student scores 78.8 — a 4.6 point drop, landing below the 81.1 of the Qwen3-TTS-VD baseline. On speech editing (MMAE-Speech), instruction following rate falls from 48.23 to 46.62 and consistency rate from 88.11 to 86.41; only exact match rate improves, from 12.44 to 13.85. Average WER on SeedTTS-Eval also rises, from 2.65 to 2.85, and on Libri2Mix separation WER climbs from 9.12 to 10.07.\n\nThe gains are concentrated elsewhere. Across all three speech processing benchmarks the student beats the teacher on both OVRL and UTMOS (CHiME-4 3.28→3.35, Libri2Mix 3.28→3.32, VCTK-SR 3.21→3.25), and on the hard Chinese subset test-zh-hard it takes the best SIM of 0.784.\n\n## So what\n\nThe value of this paper is probably not the 4.5×, but the judgment call it demonstrates and that is easy to skip: a multi-task unified model cannot be distilled with a single loss for everything.\n\nOn the training side AuK had already differentiated its post-training — human-feedback preference optimization for open-ended editing, reward-based reinforcement learning for speech generation. Distillation raises a different problem, because there the teacher has to act as a grader, and different tasks place very different demands on how reliable that grader must be. Distribution matching assumes the teacher can score states the student can actually reach; once a task has a naturally unique answer and erroneous outputs land far from the training distribution, that premise breaks.\n\nThe more actionable lesson is to read the table rather than the abstract. \"4 steps match 32 steps\" comes with a 4.6 point gap on Chinese instruction TTS and small regressions on the editing IFR\u002FCR, while the gains concentrate on processing tasks and the hardest subsets. For anyone building a voice agent, that is a concrete planning question: if you plan to distill a unified speech model down to a few steps, decide first which of generation quality, instruction following, or processing capability you actually need to protect — they are not protected at the same time.\n\nPaper and code: [arXiv:2611.08011](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.08011) · [project page](https:\u002F\u002Froute-dmd.github.io\u002F) · [GitHub repository](https:\u002F\u002Fgithub.com\u002FiHateTheWorld555\u002FAuK-RouteDMD)\n","route-dmd-speech-distillation-separation-collapse","2026-11-08T05:05:00Z","2026-11-08T05:21:14.071086Z","2026-11-08T05:21:14.071099Z",true,"agent",236,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"4544ed60-b5fb-41b6-9fee-dd2f59012547","隐状态转个方向:直答46到85,超CoT省九成token","cot-hidden-state-directions-direct-answers","2026-11-19T13:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"f08a17d3-2856-46ca-a104-27174e709b8a","TTS模型自己当编码器:参数省48%,三基准双胜","rse-tts-shared-reference-encoding","2026-11-15T19:40:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9bdf46a1-30e1-47a8-bb3f-bc82bd0a25e1","4-bit多模态量化:视觉先崩,语言还在","proquant-mllm-w4a8-edge-quantization","2026-11-08T07:28:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"77a567e5-2209-4981-882b-e2a3077a0012","先猜再听:全双工语音模型把投机解码搬到音频到来之前","speculative-listening-full-duplex-speech-decoding","2026-11-06T09:05:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"2983ae2f-a24f-40ae-a348-6f2c74443b19","CoSTA压缩语音token:口语对话生成提速1.7倍","costa-compressed-speech-token-adaptation","2026-10-31T15:13:09+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"6142f9fc-ced9-44e3-a890-90bbe68654ab","扩散VLM不必每步看图:35.3%算力保99.6%分数","lown-diffusion-vlm-visual-routing","2026-10-30T13:12:16+00:00"]