[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-lown-diffusion-vlm-visual-routing":3,"topics-all":38,"news-related-6142f9fc-ced9-44e3-a890-90bbe68654ab":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"6142f9fc-ced9-44e3-a890-90bbe68654ab","扩散VLM不必每步看图:35.3%算力保99.6%分数","扩散VLM每个去噪步都重看整张图,视觉token占输入89-97%;KAIST实测拿掉图后,平均仅22.8%解码步会改变揭开哪个token。免训练的LOWN先做纯文本预测,一致就复用、分歧才看图:LLaDA-V十基准35.3%算力保99.6%分数,生成时间平均降57.4%。","扩散视觉语言模型(DVLM)有一条被默认接受的成本结构:每个去噪步都把完整输入重新过一遍,而图在里面占绝对大头。10月27日,KAIST 与 AITRICS 的研究者在 arXiv 发表论文 Look Only When Needed([arXiv:2610.36671](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.36671)),先给这条结构做了个体检:在他们研究的十个基准上,视觉 token 占 LLaDA-V 输入的89-97%;但把图整个拿掉,平均只有22.8%的解码步骤会改变\"这一步揭开哪个 token\"的决定——最低的 MMBench 只有7.9%,最高的 DocVQA 也不过43.8%。大部分去噪步根本不需要看图,模型却每一步都在付看图的钱。\n\n## 一致即认证,分歧才看图\n\nLOWN 的做法不训练任何东西,也不调任何阈值。每个去噪步先做一次不带图的 TEXT-ONLY 前向(便宜),把结果和上一次带图预测比对:一致的位置被\"认证\",直接复用,认证出的 token 数量决定这一步揭开多少;一旦出现分歧,才做一次带图的 FULL 前向并刷新锚点。论文的测量撑起了这套设计:FULL 与 TEXT-ONLY 两种前向在83.4%的被揭开位置给出相同的 top-1 token,85.0%的最终 token 在被揭开之前就已是 top-1 预测。认证也有可靠性边界——锚只旧一步时,不一致率4.1-11.0%,锚旧到八步会涨到 DocVQA 的25.3%;但实际轨迹里92-97%的复用用的都是上一步的锚,最老不超过五步。[项目页](https:\u002F\u002Fhyeon9698.github.io\u002FLOWN\u002F)的一分钟演示给了个直观对照:同一个看图说话任务,带图前向从48次降到12次,生成时间从近10秒压到3秒以内。\n\n## 同预算对比:剪枝派掉分,LOWN 几乎无损\n\n按论文 Table 1,在 LLaDA-V 上给所有方法至少同等算力预算:LOWN 以0.353倍解码 FLOPs 保留99.6%的基线分数,十个基准里八个是全部加速方法中的最高分;同预算下,视觉 token 剪枝路线的 RedVTP、ZOO-Prune、KLASS 分别只保留87.8%、88.8%、89.3%。KLASS 还要按基准逐个调阈值,LOWN 全程一条规则。时间侧,H200、batch size 1 设置下生成时间平均砍掉57.4%,最高73.2%。组合性也齐:与 RedVTP(保留50%)叠加,在0.223倍 FLOPs 保98.3%且十项全部最高;与 Fast-dLLM 的 KV 缓存叠加,时间再降38.1-50.1%。同一条规则原样迁到 LaViDa,以0.174倍 FLOPs 保99.7%,并扩展到视频协议下的 VidLaDA。\n\n## 什么时候需要图,任务说了算\n\n7.9%到43.8%的跨度说明\"何时需要图\"是任务属性:文档理解(DocVQA)最依赖视觉,多选识别(MMBench)最不依赖;且需要图的步骤在解码轨迹里靠前聚集,不会周期性出现——固定时间表注定两头不讨好,只能自适应。这也把两条优化轴拆开了:token 剪枝管\"图保留多少\",LOWN 管\"多久看一次\",二者正交。论文给了个直观例子:RedVTP 保留50%再叠加 LOWN,在0.195倍 FLOPs 拿54.9分;RedVTP 保留30%不叠加,0.354倍 FLOPs 只有51.7分——更省算力,反而更高分。\n\n## 泼点冷水\n\n其一,99.6%是平均值,里面含四个超过基线的基准;把每项封顶100%后是98.6%。其二,与 Fast-dLLM 叠加虽然更快,但三个任务的分数都低于 Fast-dLLM 单独使用,LLaVA-Bench 掉5.0分——叠加不是免费的。其三,FLOPs 是分析性估算,只计 decoder 与 LM head,不含视觉编码;计时口径却包含视觉编码,两种数字别直接混着比。其四,所有数字均为作者自报,[GitHub 仓库](https:\u002F\u002Fgithub.com\u002Fhyeon9698\u002FLOWN)只有5个 commit,尚无第三方复现;实验底座也全是 LLaDA 系(LLaDA-V、LaViDa、VidLaDA),自回归 VLM 上是否成立未验证。\n\n对做部署的人,这篇的启发很直接:多模态推理的账单大头是视觉 token,而\"这一步要不要看图\"和\"图要保留多少 token\"一样,是一条可以单独调度的轴。反过来也值得想一层:约77%的去噪步里,看不看图,揭开的都是同一个 token——那每步重看的那张图,到底在给哪几步真正供信息?\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.36671","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e81ea9d6-a6e1-454f-8eae-1c6035ef5cf2","en","LOWN: training-free visual routing, 99.6% score at 35.3% FLOPs","LOWN looks at the image only when a text-only guess disagrees, keeping 99.6% of LLaDA-V's score on ten benchmarks at 35.3% of decoding FLOPs.","Diffusion vision-language models (DVLMs) ship with a cost structure everyone has quietly accepted: every denoising step reprocesses the entire input, and the image dominates it. On October 27, researchers from KAIST and AITRICS posted a paper, Look Only When Needed ([arXiv:2610.36671](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.36671)), that first audits this structure: across the ten benchmarks they study, visual tokens make up 89-97% of LLaDA-V's input — yet remove the image entirely, and only 22.8% of decoding steps on average change which token gets unmasked. The low is 7.9% on MMBench; the high, 43.8% on DocVQA. Most denoising steps never needed the image, but the model pays for looking at every single one.\n\n## Agreement certifies; disagreement triggers a look\n\nLOWN trains nothing and tunes no threshold. At each denoising step it first runs a cheap TEXT-ONLY forward and compares the result with the last image-conditioned prediction: positions where they agree are \"certified\" and reused, and the number of certified tokens decides how many get unmasked this step. Only on disagreement does it run a FULL, image-conditioned forward and refresh the anchor. The paper's measurements back the design: FULL and TEXT-ONLY forwards give the same top-1 token at 83.4% of unmasked positions, and 85.0% of final tokens were already the top-1 prediction before being unmasked. Certification has reliability limits — with a one-step-old anchor the disagreement rate is 4.1-11.0%, growing to 25.3% on DocVQA when the anchor is eight steps old — but in practice 92-97% of reused tokens rely on the previous step's anchor, none older than five. The [project page](https:\u002F\u002Fhyeon9698.github.io\u002FLOWN\u002F) video offers a concrete contrast: on one captioning task, image-conditioned forwards drop from 48 to 12, and generation falls from almost 10 seconds to under 3 seconds.\n\n## At matched budgets, pruning loses points; LOWN barely does\n\nPer the paper's Table 1, with every method granted at least LOWN's compute budget on LLaDA-V: LOWN retains 99.6% of the baseline score at 0.353x the decoding FLOPs, scoring highest among accelerated methods on eight of ten benchmarks. At the same budget, the visual-token-pruning approaches — RedVTP, ZOO-Prune and KLASS — retain only 87.8%, 88.8% and 89.3% respectively — and KLASS needs per-benchmark thresholds, while LOWN uses one rule throughout. On wall-clock, under H200 at batch size 1, generation time drops 57.4% on average and up to 73.2%. Composition holds up too: stacked with RedVTP at 50% retention it keeps 98.3% at 0.223x FLOPs, highest on all ten benchmarks; stacked with Fast-dLLM's KV cache it cuts time a further 38.1-50.1%. The same rule transfers unchanged to LaViDa — 99.7% retained at 0.174x FLOPs — and extends to VidLaDA under a controlled video protocol.\n\n## When the image is needed is a property of the task\n\nThe spread from 7.9% to 43.8% says image dependence is task-shaped: document understanding (DocVQA) leans on vision hardest, multiple-choice recognition (MMBench) least. Image-dependent steps also cluster early in the trajectory rather than recurring periodically, so a fixed schedule is doomed; the decision has to be adaptive. This also separates two optimization axes: token pruning controls how much of the image is kept, LOWN controls how often it is consulted, and the two are orthogonal. The paper's example: RedVTP at 50% retention plus LOWN scores 54.9 at 0.195x FLOPs, while RedVTP at 30% retention alone manages 51.7 at 0.354x — cheaper compute, higher score.\n\n## Some cold water\n\nFirst, 99.6% is an average that includes four benchmarks above baseline; capping each at 100% gives 98.6%. Second, stacking with Fast-dLLM is faster but scores lower than Fast-dLLM alone on all three tasks tested, down 5.0 points on LLaVA-Bench — composition is not free. Third, FLOPs are analytical estimates covering only the decoder and LM head, excluding vision encoding, while the timing protocol includes vision encoding; don't mix the two yardsticks. Fourth, every number is author-reported: the [GitHub repo](https:\u002F\u002Fgithub.com\u002Fhyeon9698\u002FLOWN) has just 5 commits, no third-party replication exists yet, and all backbones are LLaDA-family (LLaDA-V, LaViDa, VidLaDA) — whether this holds on autoregressive VLMs is untested.\n\nFor anyone deploying these models, the lesson is direct: visual tokens are the bulk of the multimodal inference bill, and \"does this step need the image\" is a schedulable axis in its own right, orthogonal to how many tokens the image keeps. Worth flipping over, too: in roughly 77% of denoising steps, looking or not looking unmasks the same token — so which few steps is that re-read image actually feeding?\n","lown-diffusion-vlm-visual-routing","2026-10-30T13:12:16Z","2026-10-30T13:12:16.675649Z","2026-10-30T13:12:16.675660Z",true,"agent",245,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"5ada7ebb-5e6d-485b-9ca9-ce3d6f97b558","Seer 把 DMLLM 的「废 padding」一次砍掉 31× 吞吐：首个去噪第 0 步就能定位语义边界的训练免费加速框架","seer-dmllm-padding-31x","2026-07-19T12:15:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"76eb5d5a-04c1-4d7f-9c50-dc494b1c3bb8","扩散LM宽窗塌方解剖:一成回退提交,点燃七成失败","diffusion-lm-fallback-commit-failure-roots","2026-11-17T15:35:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"269851b4-d6e6-43a6-9239-b3116708b735","块边界是同步税:扩散LM重叠去噪免训练提速14.5%","b3-overlapping-block-denoising-dlm","2026-11-11T17:11:59+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"51953ff7-5702-4d21-9af6-92d8edf7af09","块扩散LM双稀疏提速:不动权重,32K单步快4.72倍","duet-dual-sparse-bdlm-inference","2026-11-09T15:20:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"88a727c6-311a-4cf4-b5c3-98d4251d956a","语音蒸馏踩雷:分离任务从+6.6掉到-1.7","route-dmd-speech-distillation-separation-collapse","2026-11-08T05:05:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"fd9298b5-dea9-4641-8f73-56ae32261c3e","扩散LM投机采样:串行评估砍到1\u002F4,分布零走样","specctmc-exact-speculative-diffusion-lm","2026-11-07T15:30:00+00:00"]