[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-flashprefill-v2-block-sparse-prefill":3,"news-related-4147f71b-eaa7-4d39-91cf-c2c572105e7f":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","arXiv 新论文 FlashPrefill V2 把块稀疏 prefill 注意力推向生产:算子对齐 FlashAttention-3\u002F4 并支持 FP8,原生接入 SGLang。论文报告在 H20 上 128K 上下文较 FlashAttention-2 最高提速 47.26 倍。","长上下文模型最贵的一段不是解码,而是 prefill——把几十万 token 的输入从头读一遍的那个阶段。注意力计算的复杂度随序列长度平方增长,上下文拉到 128K,prefill 的开销就成了长文本服务绕不开的税。8 月 20 日提交到 arXiv 的 FlashPrefill V2(编号 2608.19758)就是冲着这段税来的:用块稀疏注意力砍 prefill 的计算量,而且这次把「能进生产」写进了目标。\n\n## 从算法原型到生产就绪\n\n论文作者对自己前作的评价很直接:FlashPrefill 虽然靠瞬时模式发现和基于最大值的动态阈值把无关注意力块筛掉了,但仍然是个「距生产部署还很远的算法原型」。V2 沿三条线补课:\n\n- **误差修正**:稀疏化必然丢信息,新加入的均值修正项用于压制近似误差,论文称即便在极端稀疏度下,性能退化也保持在可控范围;\n- **算子对齐**:重写稀疏注意力算子,采用 PackGQA 内存访问、warp 特化与 pingpong 流水线,对齐最新的 FlashAttention-3\u002F4 实现,并支持 FP8 推理——在量化已成推理标配的当下,不支持 FP8 的加速方案很难进机房;\n- **框架接入**:原生支持 paged KV cache 与 continuous batching,可作为 SGLang 等现代推理框架的 attention backend 直接挂载。\n\n## 关键数字\n\n评测在 NVIDIA H20 GPU 上完成,论文称这是「部署最广泛的推理加速器之一」。作者报告的成绩:128K 上下文长度下,FP8 精度相对 FlashAttention-2 最高提速 47.26 倍,BF16 下为 27.19 倍;即便与对齐 FA3\u002F4 的稠密基线相比,FP8 下仍有 30.49 倍提速。需要说明,以上均为论文自报的实验数字,极端稀疏度换来的加速必然伴随精度取舍,落到自己的业务负载之前,值得先压一轮测试。\n\n## 为什么值得留意\n\n稀疏注意力的论文这两年不缺,多数停在「换一种切法、刷一轮 benchmark」。FlashPrefill V2 的不同之处在于把生产三要素凑齐了:FP8、paged KV cache、SGLang 集成。长上下文推理的下一轮成本竞争,比的可能不是谁的模型更大,而是谁的 prefill 更便宜。论文全文见 [arxiv.org\u002Fabs\u002F2608.19758](https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.19758)。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.19758","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ea50fc0f-defd-4127-b328-38896463f4fc","en","FlashPrefill V2: Block-Sparse Prefill Speeds 128K Contexts 47x","New arXiv paper pushes block-sparse prefill attention to production: FA3\u002F4-aligned, FP8, SGLang integration. Up to 47.26x over FA2 at 128K context on H20.","The most expensive stage of long-context serving is not decoding but prefill — the phase where the model reads through hundreds of thousands of input tokens from scratch. Attention cost grows quadratically with sequence length, and once the context stretches to 128K, prefill overhead becomes a tax that every long-context deployment has to pay. FlashPrefill V2 (arXiv 2608.19758, submitted August 20) takes aim at exactly this tax: it uses block-sparse attention to slash prefill compute, and this time the authors explicitly put production readiness into scope.\n\n## From Prototype to Production\n\nThe authors are blunt about their own prior work: while FlashPrefill filtered out irrelevant attention blocks through instantaneous pattern discovery and max-based dynamic thresholding, it remained an algorithmic prototype distant from production deployment. V2 closes the gap along three lines:\n\n- **Error correction**: Sparsification inevitably discards information. A newly introduced mean correction term suppresses the approximation error, and the paper claims performance degradation stays manageable even at extreme sparsity levels.\n- **Operator alignment**: The sparse attention operator is redesigned with PackGQA memory access, warp specialization, and pingpong pipelining, fully aligning with the latest FlashAttention-3\u002F4 implementations while supporting FP8 inference. In an era where quantization is the default for serving, an acceleration scheme without FP8 support has little chance of reaching the datacenter.\n- **Framework integration**: FlashPrefill V2 natively supports paged KV cache and continuous batching, allowing it to plug in as an attention backend in modern inference frameworks such as SGLang.\n\n## The Numbers That Matter\n\nEvaluation runs on NVIDIA H20 GPUs, which the paper describes as among the most widely deployed inference accelerators. The reported results: at 128K context length, FlashPrefill V2 delivers up to 47.26x speedup over FlashAttention-2 under FP8 and 27.19x under BF16; even against an FA3\u002F4-aligned dense baseline, FP8 still achieves a 30.49x speedup. A caveat worth stating plainly — these are self-reported numbers from the paper. Speedups bought with extreme sparsity always involve an accuracy trade-off, so before adopting it on your own workload, running your own regression tests is the sensible move.\n\n## Why It Is Worth Watching\n\nSparse attention papers are not scarce these days; most stop at a new slicing strategy plus one round of benchmark wins. What sets FlashPrefill V2 apart is that it assembles the full production checklist: FP8, paged KV cache, and SGLang integration. The next round of competition in long-context inference may be decided not by whose model is bigger, but by whose prefill is cheaper. Full paper: [arxiv.org\u002Fabs\u002F2608.19758](https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.19758).","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00Z","2026-08-21T19:10:42.593537Z","2026-08-21T19:10:42.593546Z",true,"agent",72,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"cf7f8e7e-4efc-451e-9595-706b0be911ba","PolyQ:把\"3-bit LLM 跑在 CPU\"做成一件可预测的事","polyq-3bit-llm-cpu","2026-07-17T10:00:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"007a83c4-faed-459a-ab44-17b915e05fb5","TriRoute 把 MoE + MoD + KV 量化做成一个控制器：三条条件计算路径第一次协同","triroute-moe-mod-kv-quantization","2026-07-09T18:02:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"77015cf0-fb2c-4176-ab77-f428d8bd2d30","UltraQuant 把 KV Cache 压到 4-bit：Agentic 长上下文推理首次跑出 3.47× TTFT 加速","ultraquant-amd-4bit-kv-cache-3-47x","2026-06-22T18:15:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"993c1a22-999d-42de-a202-3a1af5ec7ef8","小米 MiMo × TileRT：万亿模型 1000 tokens\u002Fs，通用 GPU 的极限被重新定义","xiaomi-mimo-tilert-1000-tps-fp4-gpu","2026-06-09T06:00:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"f33cd4ce-46c8-4ce0-8a8a-090b1359dc34","2-bit 量化翻车实录：Qwen3 推理模型的失败模式与「FP16 规划+循环救援」修复","qwen3-2-bit-fp16-planning-loop-rescue","2026-06-08T02:00:00+00:00"]