[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-dflash-diffusion-speculative-tpu-3x-vllm":3,"topics-all":33,"news-related-e580f858-d0c3-420d-8825-362bc0004845":52},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":20,"news_slug":26,"published_at":27,"created_at":28,"modified_at":29,"is_published":30,"publish_type":31,"image_url":13,"view_count":32},"e580f858-d0c3-420d-8825-362bc0004845","DFlash：扩散式投机解码让TPU推理提速3倍","自回归LLM生成速度受限？草稿模型猜token仍是顺序计算瓶颈。UCSD联合Google在arXiv发表DFlash工作：将扩散模型的block生成思路引入投机解码，在TPU v5p上实现平均3.13倍加速，数学与代码任务最高达6倍，已集成进vLLM TPU生态。投机解码的软肋：传统投机解码用小模型猜、大模型验证，但草稿阶段本身仍需逐token生成——猜K个token就得跑K步自回归。模型越大、序列越长，这个嵌套瓶颈越明显。DFlash的核心是把扩散模型中一次性生成整块的思想迁移到token领域，草稿阶段并行生成一整块draft tokens，验证阶段以block为单位批量处理。工程上依赖双缓存架构、2的幂次填充优化CPU-TPU数据传输、状态同步防止序列长度膨胀。TPU v5p的K-Flat发现：验证成本对block size在16到1024之间几乎不变，提升草稿质量比增大block size更划算。相对EAGLE-3，DFlash实现2.29倍端到端加速，证明非自回归草稿生成在LLM推理中完全可行，有望成为大模型部署的标配优化技术。","https:\u002F\u002Fdevelopers.googleblog.com\u002Fsupercharging-llm-inference-on-google-tpus-achieving-3x-speedups-with-diffusion-style-speculative-decoding\u002F","3318cb52-f01e-4c9e-a34a-5dbc9fa986f2",[10,14,17],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[21],{"id":22,"lang":23,"title":24,"summary":25,"content":13},"3770dacf-dd15-463b-aaf9-3200e14f1bff","en","DFlash: diffusion speculation triples TPU inference","Autoregressive LLM generation speed limited? Draft-model token guessing is still a sequential-compute bottleneck. UCSD and Google jointly published DFlash on arXiv: bringing the block-generation idea of diffusion models into speculative decoding, achieving an average 3.13× speedup on TPU v5p, up to 6× on math and code tasks, already integrated into the vLLM TPU ecosystem.\n\n**The soft spot of speculative decoding**: traditional speculative decoding uses a small model to guess and a large model to verify, but the draft phase itself still requires sequential token generation — guessing K tokens means running K autoregressive steps. The larger the model, the longer the sequence, the more obvious this nested bottleneck becomes. DFlash's core is migrating the \"generate a whole block at once\" idea from diffusion models into the token domain: the draft phase generates a whole block of draft tokens in parallel, the verification phase handles them in block units. Engineering-wise it relies on dual-cache architecture, power-of-2 padding to optimize CPU-TPU data transfer, and state synchronization to prevent sequence-length inflation. **TPU v5p's K-Flat finding**: verification cost is nearly constant for block sizes between 16 and 1024, making improving draft quality more cost-effective than increasing block size. Compared to EAGLE-3, DFlash achieves 2.29× end-to-end speedup, proving non-autoregressive draft generation is fully viable in LLM inference, with the potential to become a standard optimization for large-model deployment.","dflash-diffusion-speculative-tpu-3x-vllm","2026-05-10T13:01:00Z","2026-05-10T13:11:37.809095Z","2026-08-19T02:08:40.142862Z",true,"agent",150,[34,43],{"slug":35,"tag_slug":35,"title_zh":36,"title_en":37,"intro_zh":38,"intro_en":39,"id":40,"is_active":30,"created_at":41,"modified_at":42},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":44,"tag_slug":44,"title_zh":45,"title_en":46,"intro_zh":47,"intro_en":48,"id":49,"is_active":30,"created_at":50,"modified_at":51},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":53},[54,59,64,69,74,79],{"id":55,"title":56,"news_slug":57,"published_at":58},"c3814f7d-2649-4660-a798-28fb03aa2b6d","SwitchSD 让投机解码学会「该抄才抄」:读内部信号,EAGLE3 之上再快 15%","switchsd-copy-intent-speculative-decoding","2026-09-20T23:09:25+00:00",{"id":60,"title":61,"news_slug":62,"published_at":63},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"813ad679-51dd-43d7-afcc-0baf48d2ef5f","When2Think:推理模型该想多久,先看题有多难","when2think-difficulty-aware-length-control","2026-09-19T19:08:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"9a3cd449-e29a-4730-814b-f1be5c2685c6","复旦FFD让Flash Attention退役？11.6× kernel提速把长上下文推到256K","fudan-ffd-long-context-attention-sparsity","2026-09-15T07:15:46+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00"]