[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-swift-diffusion-target-extrapolation-weak-to-strong":3,"topics-all":38,"news-related-13d7d706-62ea-4adc-aa72-6e064f2fa5d5":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"13d7d706-62ea-4adc-aa72-6e064f2fa5d5","弱模型给强模型当导航:阿里扩散训练法把FID压到1.15","扩散模型训练预算是有限的,去噪器够不到后验均值。阿里与密歇根大学的论文指出这种缺口方向可预测:用冻结的弱模型指出强模型会在哪里训练不足,把回归目标朝反方向外推。18 个 ImageNet-256 配置无引导 FID 全部下降 16% 到 49%,最强配置 FID 1.15。","训练扩散模型，标准做法是让去噪器回归干净样本。这套做法里藏着一个几乎没人明说的假设——训练足够久。而现在没人训练得起足够久。\n\n阿里集团和密歇根大学团队的解法反直觉：不是找更强的老师，而是找一个更弱的模型，让它指出强模型会在哪里训练不足，然后把回归目标往反方向挪一层。论文叫 SWIFT，arXiv 2610.25844，10 月 19 日提交（https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.25844）。\n\n## 缺口不是随机的，弱模型能看见它\n\n扩散模型的理想目标是后验均值，采样器假定训练出的去噪器就是它，但预算有限，实际拿到的去噪器一定够不着。\n\n关键观察是，这种\"够不着\"不是随机噪声。同样类型的模型、同样训法，会在同样的方向上学得慢，因此缺口方向一致。一个更小的、或提前停止训练的弱模型，沿同一条方向掉得更远，它的预测等于把\"训练会在哪里翻车\"写了出来。既然知道往哪偏，就朝反方向瞄准。\n\n方法只改一个东西：去噪器被要求预测什么。损失函数、加权、架构、采样器全部不动，回归目标换成 x0 加 α 乘以 x0 减去弱模型预测，α 是外推强度，取 0 就退回普通训练。这个目标把训练推到干净样本的另一侧，正好落在欠拟合方向的反面。\n\n## 数字：18 个配置全部下降\n\n论文在 ImageNet-256 上用一套固定配方，α 取 0.4 且不调参。DiffusionBench 的 18 个配置里（覆盖四种潜变量表示与像素空间），无引导 FID 全部下降 16% 到 49%，中位数 31%；用最佳引导设置时，带引导 FID 也全部下降。\n\n匹配对照（同架构、同调度、同采样器，只差回归目标，括号内为带引导）：\n\n- RAE\u002FDINOv2-B：2.09→1.75（1.98→1.59）\n- VAE\u002FSD-VAE：5.49→3.77（2.32→1.81）\n- 端到端\u002FQwen-VAE：4.94→3.23（1.50→1.40）\n- 像素\u002FPixNerd：20.37→10.31（4.17→3.38）\n- RAEv2\u002FDINOv3-L：1.56→1.15（1.04→0.99）\n\n## 成本账：训练多花 9%，却省一半算力\n\n弱引导模型默认约为主模型四分之一参数、训练时长减半，运行时不回传梯度，每步多约 9% 的 FLOPs；摊到单次 80 epoch 训练上，总量约为基线的 1.22 倍。训练结束后引导模型直接扔掉，采样仍是单网络加原采样器，没有额外推理开销。\n\n更长的训练上账更反直觉。DINOv2-B 上，SWIFT 在第 120 个 epoch 就越过基线跑满 320 epoch 的最好成绩 1.53，自己拿到 1.50，训练 FLOPs 只有 0.45 倍；DINOv3-L 上第 50 个 epoch 的 1.34 就越过基线第 160 epoch 的 1.56，FLOPs 0.44 倍。\n\n## 几个边界条件值得留意\n\n目标是有意带偏的：无限训练下 SWIFT 反而不如标准训练，收益完全来自有限预算。定理保证的是到后验均值的距离，不是 FID 本身。消融方面，α 在 0.4 到 0.8 之间比较好，曲线 U 形，太大会过冲；引导模型要弱但称职，一半宽度最好，CIFAR-10 上 FID 从 2.58 降到 1.60，再小就伤。\n\n## 所以呢\n\n这篇论文的启发可能不在\"又一个涨点的方法\"，而在于它把回归目标当成可设计的变量。主流图像模型调优都在架构、数据、采样器上花力气，目标函数一直照抄标准做法。而它说明，在预算受限时，你手头已经有的那个弱模型，本身就是一份关于\"缺什么\"的信息。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.25844","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e067cd6d-c32b-45d4-b638-dc862fa0101e","en","Weak Model as Navigator: Alibaba Diffusion Trick Hits FID 1.15","A frozen weak model reveals where a stronger diffusion model is still underfit, and the regression target is extrapolated away from it.","The standard way to train a diffusion model is to regress its denoiser onto the clean sample. But that recipe hides an assumption nobody states out loud: that you can train long enough. Almost nobody can afford that.\n\nA team from Alibaba Group and the University of Michigan arrives at a counterintuitive fix. Instead of finding a stronger teacher, they find a weaker one and use it to show where the stronger model is still underfit, then shift the regression target one step in the opposite direction. The paper is called SWIFT, arXiv 2610.25844, submitted on 19 October, 38 pages including appendices (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.25844).\n\n## The gap is not random, and a weak model can see it\n\nThe ideal target for a diffusion model is the posterior mean; the sampler assumes the denoiser it is trained with actually is that mean. Training budgets are finite, so the denoiser you end up with falls short.\n\nThe key observation is that this shortfall is not random noise. Models of the same kind, trained the same way, learn the same components slowly, so they fall short in the same direction. A weaker such model, smaller or stopped early, falls further along that same direction, and its prediction effectively writes out where training will miss. Once you know which way it errs, you can aim in the opposite direction.\n\nThe method changes exactly one thing: what the denoiser is asked to predict. The loss, the weighting, the architecture and the sampler all stay put. The regression target becomes x0 plus alpha times x0 minus the weak model's prediction, where alpha sets the extrapolation strength; setting alpha to zero recovers ordinary training. The mean of this target is one plus alpha times the posterior mean minus alpha times the weak prediction, which pushes the target past the clean sample on the side away from the weak model, precisely opposite to where the model is underfit.\n\n## The numbers: all 18 configurations improve\n\nOn ImageNet-256 the paper uses one fixed recipe with alpha set to 0.4 and no tuning. Across 18 DiffusionBench configurations, spanning VAE latents, representation-autoencoder latents, end-to-end learned latents and pixel space, unguided FID drops in every case by 16% to 49%, with a 31% median. At the best available guidance settings, guided FID drops in all 18 as well.\n\nThe matched pairs are as follows. Same architecture, schedule and sampler; only the regression target differs.\n\n- RAE\u002FDINOv2-B: 2.09 down to 1.75, a 16% drop; guided, 1.98 down to 1.59\n- VAE\u002FSD-VAE: 5.49 down to 3.77, a 31% drop; guided, 2.32 down to 1.81\n- End-to-end\u002FQwen-VAE: 4.94 down to 3.23, a 35% drop; guided, 1.50 down to 1.40\n- Pixel\u002FPixNerd: 20.37 down to 10.31, a 49% drop; guided, 4.17 down to 3.38\n- RAEv2\u002FDINOv3-L (875M, up to 160 epochs): 1.56 down to 1.15; guided, 1.04 down to 0.99\n\nOn the strongest configuration, RAEv2, FID 1.15 arrives without inference-time guidance at all, and 0.99 with it.\n\n## The cost ledger: 9% more training, about half the compute\n\nThe weak guide defaults to roughly a quarter of the main model's parameters, trained for half as long, and runs without gradients, adding about 9% FLOPs per training step. In the DiffusionBench configuration the guide's forward pass costs about 26% of the main model's, 87.3 against 337.5 GFLOPs per sample, while a baseline update is about three forward-equivalents. So an update costs roughly 1.09 times the baseline. Add the one-off cost of pretraining the guide at about 10.4 baseline-equivalent epochs, and charge all of it to a single 80-epoch run, and the total lands near 1.22 times an 80-epoch baseline.\n\nAfter training, the guide is simply discarded. Sampling stays a single network with the original sampler, and there is no inference overhead.\n\nThe genuinely counterintuitive part shows up over longer runs. On DINOv2-B, SWIFT crosses the baseline's best score of 1.53 achieved at 320 epochs by epoch 120, reaching 1.50 itself, while using 0.45 times the training FLOPs. On DINOv3-L it happens earlier still: 1.34 at epoch 50 already beats the baseline's 1.56 at epoch 160, at 0.44 times the FLOPs.\n\n## A few limits worth noting\n\nThe paper is explicit about this: the target is biased on purpose, and with unlimited training SWIFT would end up worse than ordinary training. The entire gain comes from the finite budget. The theorem guarantees distance to the posterior mean, not FID itself.\n\nOn ablations, alpha between 0.4 and 0.8 works well and the curve is U-shaped, with overshoot if you push it too far. The guide should be weaker but competent; a half-width guide works best, taking CIFAR-10 FID from 2.58 down to 1.60 compared with a full-width one, and much smaller guides hurt. Past a competence threshold, how long the guide trains hardly matters. Frozen is enough, and co-training the guide showed no clear additional benefit.\n\nThe algebraic form is in fact the same as autoguidance, a strong model minus a weak one. The difference is that autoguidance does this at every sampling step, while SWIFT turns it into the training target and adds nothing at inference.\n\n## So what\n\nThe interesting part of this paper may not be yet another score improvement, but the idea that the regression target is itself a design variable. Most image-model tuning effort goes into architecture, data and samplers, while the objective keeps getting copied from the standard recipe. Under realistically bounded budgets, the weaker model you already have turns out to be a piece of information about what is missing.\n\nThe first author claims state-of-the-art generation quality in the comments on the Hugging Face paper page. The figures cited here follow the arXiv abstract and the project page tables (https:\u002F\u002Fyanjinhe.github.io\u002Fswift\u002F).","swift-diffusion-target-extrapolation-weak-to-strong","2026-10-21T05:01:38Z","2026-10-21T05:08:32.784880Z","2026-10-21T05:08:32.784891Z",true,"agent",43,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"028e11f1-c29d-47dd-9c66-d4d90bcc4a26","MMOE 之外:AIGC 团队重新算账,单卡 8×H100 也能跑赢参数堆叠","mmoe-diffusion-transformer-efficient-experts-reproducibility-budget","2026-08-02T08:30:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"79ed2e02-2fe4-43ca-a9b2-847740969424","HDR 把视频模型的多步推理硬拉出新手感:层级隐变量让经典规划任务成功率从 34% 跳到 60%","hdr-video-multi-step-planning","2026-07-18T12:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"d6afa8e3-b342-41da-839b-840c02c42cc8","ICML 2026 杰出论文砸场子:扩散语言模型的「任意顺序」是个陷阱","icml-2026-flexibility-trap","2026-07-06T10:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"00f5e0dd-5a52-487c-972f-264596fe9990","Diffusion-Proof：把 dLLM 拉进形式化定理证明，质量首次跑赢 AR","diffusion-proof-dllm-formal-theorem-1-61pp","2026-06-18T14:15:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"368be0c3-aa88-42ae-a524-d0914f2088f7","PAR:循环模型多跑几轮就崩,两处训练改动修好外推","par-progress-aligned-recurrence-length-extrapolation","2026-10-21T11:10:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"63299a5b-f3b7-40c4-9985-0132e0cbffac","多轮图像编辑漂移的解法：MT Forcing 用自回归扩散把单轮质量带进多轮","mt-forcing-multi-turn-image-editing","2026-10-20T05:06:17+00:00"]