[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-flash-dllm-io-aware-kv-cache":3,"topics-all":38,"news-related-8b50b8ec-0249-420f-a091-0d8b7efe6a18":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"8b50b8ec-0249-420f-a091-0d8b7efe6a18","Flash-dLLM 免训加速扩散语言模型:最高148倍","MBZUAI 团队开源 Flash-dLLM:用 IO 感知融合 KV 缓存内核与自起草自验证并行解码,让 LLaDA-1.5 免重训跑到每秒 148-211 token,比无缓存贪心解码快 22-148 倍,单卡 A100 显存占用比 Fast-dLLM 低约 48%。","扩散语言模型(dLLM)按块并行去噪生成文本,天然适合并行解码,但推理效率一直是走向生产的绊脚石。9 月 22 日,MBZUAI VILA Lab 在 arXiv 放出 Flash-dLLM(arXiv:2609.26796),无需重训、即插即用,把 LLaDA-1.5 吞吐拉到 148-211 token\u002Fs,比无缓存贪心解码最快提速 148.2 倍,代码以 Apache-2.0 开源。\n\n## 瓶颈不在算力,在内存 I\u002FO\n\n论文的第一个判断反直觉:KV 缓存启用后,dLLM 推理的主导瓶颈不是 attention 计算也不是解码策略,而是 GPU 显存 I\u002FO——缓存反复读写、并行验证反复搬运数据,算力大量时间在等数据。现有方法通常孤立研究 KV 缓存或并行解码,恰好漏掉这个联合 I\u002FO 瓶颈。RTX 3090 微基准里,融合缓存管线每层提速 1.37 倍;更关键的观察是,中间层 top-32 高注意力已解码 token 就吃掉约 50% 的注意力——绝大多数缓存位置无需每次重算。\n\n## Flash-Cache:融合内核加选择性刷新\n\n第一件武器 Flash-Cache 是 I\u002FO 感知的融合 KV 缓存内核:QKV 投影、RoPE、缓存写入融合进同一个 Triton kernel,中间 key\u002Fvalue 不落显存、直接写进缓存,长短不一的 query 用块表调度。刷新由注意力引导:每步只重算新解码 token,外加固定预算内按注意力重要性挑出的高影响力 token,其余位置全部吃缓存。整套设计零训练、零改动。\n\n## Flash-Verify:自己给自己当验稿员\n\n第二件武器 Flash-Verify 解决验证问题,不需要外挂草稿模型——dLLM 自己既当起草者又当验证者。高置信 token 按阈值直接接受;剩余候选以「双视图」进入验证序列,一份保留草稿值、一份填 [MASK],因果掩码让位置看不到自己的草稿、却能看到前面的草稿;两个视图预测一致且掩码侧置信度达标才接受。作者报告这招让每步接受的 token 数翻了约一倍。\n\n## 跑分好看,但别当成无损加速\n\n单卡 A100 80GB、LLaDA-1.5 基座上,Flash-Cache 加 Flash-Verify 在论文 Table 1 的八项设置中全部拿到最高吞吐:GSM8K-512 跑到 210.6 token\u002Fs(81.0 倍),MBPP-512 跑到 148.2 token\u002Fs(148.2 倍),整体提速 22.3-148.2 倍;对论文所称此前最强基线 Elastic-Cache,GSM8K-512 快 5.1 倍、HumanEval-512 快 11.0 倍,对置信度感知解码吞吐高 23.5%-45.0%。精度面,GSM8K-512 拿到 83.02%,精度吞吐双第一。内存侧,batch 16 时约 26GB,比 Fast-dLLM 的约 50GB 省约 48%;后者 batch 24 就内存溢出,Flash-dLLM 能扩到 batch 32。README 也诚实备注:HumanEval-512 上置信度感知配置 42.07% 高于 Flash-Verify 的 40.24%,代价是吞吐更低——按精度-吞吐平衡选配置,别把加速当普遍无损。\n\n## 所以呢\n\nFlash-dLLM 的价值不在某个单一倍数,而在把「I\u002FO 感知」这个系统层视角带进扩散语言模型推理:缓存与并行解码必须联合设计,真瓶颈才会暴露。对推理框架开发者和 dLLM 团队,这是可直接 fork 的 Apache-2.0 起点;对旁观者,这是 dLLM 从论文走向可部署的信号。参考:arxiv.org\u002Fabs\u002F2609.26796 与 GitHub VILA-Lab\u002FFlash-dLLM。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.26796","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7b67033c-19e6-4052-a626-e681bba64c7a","diffusion",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ebf9d1c3-3d89-45bd-b396-402ed0ccf71b","en","Flash-dLLM: 148x training-free speedup for diffusion LLMs","Flash-dLLM pairs an IO-aware KV cache with self-verifying parallel decoding: LLaDA-1.5 hits 148-211 tok\u002Fs, up to 148x faster, with ~48% less memory.","Diffusion language models (dLLMs) are often pitched as the next challenger to autoregressive LLMs: text is generated by iterative block-wise denoising, which should in principle make parallel decoding natural. Yet actual deployment has been held back by inference efficiency. On September 22, MBZUAI's VILA Lab released Flash-dLLM on arXiv (2609.26796), a training-free acceleration framework that pushes LLaDA-1.5 to 148-211 tokens per second — up to 148.2x faster than greedy decoding without caching — with the code open-sourced under Apache-2.0.\n\n## The bottleneck is memory I\u002FO, not compute\n\nThe paper's first claim is counter-intuitive: once KV caching is enabled, the dominant bottleneck in dLLM inference is neither attention compute nor the decoding strategy, but GPU memory I\u002FO — repeated cache reads and writes, plus repeated transfers for parallel token verification, leave compute waiting on data. Prior acceleration work typically studied KV caching and parallel decoding in isolation, missing exactly this joint I\u002FO bottleneck. In an RTX 3090 microbenchmark the authors measure a 1.37x per-layer speedup from the fused cache pipeline, and a second observation does the strategic work: the top-32 most-attended decoded tokens capture roughly 50% of attention in middle layers — meaning most cached positions never need to be recomputed at all.\n\n## Flash-Cache: a fused kernel with selective refresh\n\nThe first component, Flash-Cache, is an I\u002FO-aware fused KV-cache kernel: QKV projection, RoPE, and cache writes are fused into a single Triton kernel, intermediate keys and values never touch high-bandwidth memory but are written straight into the cache, and variable query lengths within a batch are handled through block-table scheduling. The refresh policy is attention-guided: each denoising step recomputes only newly decoded tokens plus a fixed-budget set of high-influence tokens selected by attention importance, while everything else is served from cache. None of this requires retraining or touching the upstream model.\n\n## Flash-Verify: the model verifies its own drafts\n\nThe second component, Flash-Verify, tackles verification for parallel decoding without any auxiliary drafter — the dLLM serves as both drafter and verifier. High-confidence tokens are accepted directly by a confidence threshold; remaining candidates are ordered by draft confidence and enter the verification query twice, once keeping the draft value and once as a [MASK], under a causal attention mask that prevents a position from seeing its own draft while allowing earlier drafts to provide context. A candidate is accepted when the draft view and the mask view agree and the mask-view confidence clears the threshold. The authors report this roughly doubles the number of tokens accepted per step.\n\n## Strong numbers, but not a free lunch\n\nOn a single A100 80GB with LLaDA-1.5, Flash-Cache + Flash-Verify posts the highest throughput across all eight benchmark\u002Flength settings in the paper's Table 1: GSM8K-512 reaches 210.6 tok\u002Fs (81.0x) and MBPP-512 reaches 148.2 tok\u002Fs (148.2x), with the overall speedup range at 22.3x-148.2x over uncached greedy decoding. Against the baseline the paper calls the prior strongest, Elastic-Cache, it is 5.1x faster on GSM8K-512 and 11.0x on HumanEval-512, and throughput runs 23.5%-45.0% higher than confidence-aware decoding. On accuracy, GSM8K-512 hits 83.02%, the best score in that setting, taking both the accuracy and throughput crowns. Memory-wise, batch 16 costs about 26GB versus roughly 50GB for Fast-dLLM — around 48% lower — and where Fast-dLLM runs out of memory at batch 24, Flash-dLLM scales to batch 32. Credit where due: the README is refreshingly honest that Flash-Verify is not the most accurate configuration everywhere — on HumanEval-512, confidence-aware Flash-Cache reaches 42.07% versus 40.24% for Flash-Verify, at lower throughput. The official guidance is to pick a configuration by accuracy-throughput balance rather than treating the speedup as universally lossless.\n\n## So what\n\nFlash-dLLM's real contribution is not any single multiplier but the systems perspective it imports: caching and parallel decoding for diffusion LLMs have to be co-designed before the I\u002FO bottleneck even becomes visible. For inference-framework developers and dLLM teams it is a ready-to-fork Apache-2.0 starting point; for everyone else, it is a concrete signal that diffusion language models are being pulled from paper territory into deployable territory. References: [arXiv:2609.26796](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.26796) and [GitHub: VILA-Lab\u002FFlash-dLLM](https:\u002F\u002Fgithub.com\u002FVILA-Lab\u002FFlash-dLLM).","flash-dllm-io-aware-kv-cache","2026-09-23T15:10:00Z","2026-09-23T15:10:04.000703Z","2026-09-23T15:10:04.000711Z",true,"agent",1548,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"6ede1566-74c5-4bdd-9dd5-6d3f118db876","投机解码的重复陷阱:D-Loop 两轮循环破局","d-loop-diffusion-speculative-decoding","2026-10-06T15:20:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"8914cfad-53bc-4f76-bb06-8579d23cfa5d","AR 大模型外挂 diffusion 权重:Uno 免草稿模型实现 3 倍无损解码加速","uno-diffusion-augmented-llm-speedup","2026-09-08T13:30:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"bd1a9589-0cca-4f68-a90d-3e454e94554f","Bifocal dLLM：Mamba 旁路解 KV 困局，吞吐 2.4×–12.9×","bifocal-dllm-r2lm-mamba-qwen3-1-7b","2026-06-29T10:08:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"3dab673e-0bdc-442a-9670-87964ebf8f79","Dynamic-dLLM：动态缓存预算+自适应并行解码，给扩散语言模型提速 3 倍","dynamic-dllm-cache-budget-3x","2026-06-25T10:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"0b744f65-a3d5-47d8-84d1-eb25a7a2798e","FMLM+ 把扩散语言模型的「自纠错」解锁：32× 更少 NFE 匹配离散基线","fmlm-plus-posterior-refinement-32x-nfe","2026-06-25T02:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"80bc5e25-d24d-45a1-9c2b-534ebcae39f9","腾讯 WeDLM 开源：让扩散 LLM 在标准因果注意力下跑出 3-6× vLLM 加速","tencent-wedlm-diffusion-llm-causal-3-6x","2026-06-16T20:00:00+00:00"]