[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-fudan-ffd-long-context-attention-sparsity":3,"topics-all":35,"news-related-9a3cd449-e29a-4730-814b-f1be5c2685c6":54},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"9a3cd449-e29a-4730-814b-f1be5c2685c6","复旦FFD让Flash Attention退役？11.6× kernel提速把长上下文推到256K","复旦Qiu Xipeng团队把Flash Attention的内存墙拆了——把\"选哪些块要算\"和\"算这些块\"合并成同一个融合kernel,免训练免改模型,长上下文解码kernel级最快11.6×、端到端2.37×,256K上下文仍保RULER\u002FLongBench精度。","长上下文 LLM 这两年的故事,本质是在跟两块墙搏斗:显存墙靠 KV cache 压缩、FlashAttention 的 IO 优化撑住;算力墙靠 chunked prefill、投机解码、MoE 来分摊。到了 256K 这种长度,FlashAttention 的\"先扫一遍找 top-k 关键块、再算这些块\"两段式流程暴露一个尴尬事实——决定要算哪些块本身,就要把所有 query-key 扫一遍,内存开销跟不解码的全注意力几乎一样。ICML 2026 上复旦 Qiu Xipeng 团队 \"Faster Than Flash\"(arXiv:2609.00097),把这个看似不动的天花板又往上顶了一截。\n\n## 一份\"老问题\"的硬约束\n\n长上下文解码的主流路径是稀疏注意力:既然 query-key 矩阵只有少数块真正重要,何必都算?问题在于怎么\"知道\"哪些块重要。最朴素的做法是用 query-block 历史最大注意力分数当\"打分牌\",但这块牌要外挂元数据、要 GPU 全局同步,在 H100 这种带宽吃紧的卡上,这块牌本身的读写就足以把后面的稀疏计算收益吃光。Faster Flash Decoding(FFD)的第一刀:把\"selector\"和\"computer\"塞进同一个完全融合的 kernel,中间不留任何元数据同步。selector 怎么工作?靠低比特量化过的 query 直接在块上扫描,scan 的结果当场就喂给 attention 计算——既不用写回全局内存,又跳过 top-k 截断需要的那次全量排序。\n\n## top-delta:把\"全局同步\"也拆掉\n\nFFD 的第二个关键设计是 top-delta 策略。当 query 在不同上下文段落里分布极不均匀时(比如 256K 里只关心最后 4K 的事实),传统做法要么\"全选\"浪费算力,要么\"全阈值\"漏掉长程依赖。top-delta 改为\"保留分数最高的 d 个块 + 跟最高分差距在 δ 内的所有块\",每个 query 独立决定 d,本地决策,无需 GPU-wide barrier。这条思路在 RULER 和 LongBench 上跑下来,关键检索类任务精度几乎不掉,代价是 scan 的工作量跟着上下文长度线性增长,但因为融合进了 kernel,反而被带宽优化吸收。\n\n## 数字层面的承诺与边界\n\n按论文数据,FFD 单 kernel 层达到最高 11.6× 加速,端到端 2.37×,上下文推到 256K。最关键是\"训练-free + plug-and-play\":不重训、不改权重、装到任何已训练好的 LLM 上即用,代码已开源在 github.com\u002Fqluoluo\u002Ffaster-flash-decoding。但边界要标:kernel 级 11.6× 是峰值不是平均;论文只在 RULER 和 LongBench 两个长上下文基准验证精度,真实业务(代码库理解、长视频字幕、agent 跨页操作)是否保精度,得跑过才知道;FFD 吃显存换带宽,如果同时叠 KV cache 压缩,在 H100 之外的老卡能不能稳,也是部署前要测的。\n\n## 所以呢\n\n长上下文赛道的下一步,大概率是\"Faster Than X\"这条线:FlashAttention 把 IO 优化做到 3.0,后续拼的是 selector-computer 融合、扫描-计算复用、动态分布自适应这三件事。FFD 给出的不是\"又一个 2×\",而是把以前默认要做的事直接砍掉——这种架构层面的减法,比纯算 kernel 优化更值得被同行盯。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.00097","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"0d6d4f2e-1d2d-453a-94d8-05b6e3c562fb","en","Faster Than Flash: Fudan FFD Hits 11.6x Kernel Speedup, Pushes Long-Context Decoding to 256K","Fudan's Qiu Xipeng group breaks Flash Attention's memory wall by fusing the block selector and the attention computer into a single kernel — training-free and plug-and-play. Long-context decoding reaches up to 11.6x kernel speedup and 2.37x end-to-end throughput, while holding RULER \u002F LongBench accuracy at 256K context.","The story of long-context LLMs over the past two years has essentially been a fight against two walls: the memory wall, kept at bay by KV-cache compression and FlashAttention-style IO optimization; and the compute wall, tamed by chunked prefill, speculative decoding, and MoE. But at 256K context length, the two-stage flow in FlashAttention — \"scan once to find the top-k critical blocks, then compute those blocks\" — exposes an awkward truth: deciding which blocks to compute already requires scanning every query-key pair, so the memory overhead is essentially the same as full uncompressed attention. A paper titled \"Faster Than Flash\" (arXiv:2609.00097) from Qiu Xipeng's group at Fudan, accepted at ICML 2026, pushes that seemingly immovable ceiling a notch higher.\n\n## The Hard Constraint on an Old Problem\n\nSparse attention is the mainstream path for long-context decoding: if only a few blocks of the query-key matrix actually matter, why compute all of them? The catch is figuring out which blocks matter. The naive approach uses each query's historical max attention score per block as a \"scoring tag\" — but that tag requires externalized metadata and GPU-wide synchronization. On bandwidth-starved cards like the H100, the read\u002Fwrite traffic for the tag alone eats most of the sparse-compute savings. Faster Flash Decoding (FFD) takes a first cut: it stuffs the selector and the computer into a single fully fused kernel, with no metadata sync in between. How does the selector work? A low-bit-quantized query scans blocks on the fly, and the scan output is fed straight into the attention compute — no round trip to global memory, and no global sort for top-k truncation.\n\n## top-delta: Tearing Down \"Global Synchronization\" Too\n\nFFD's second key design is the top-delta strategy. When queries are distributed very unevenly across long context segments (think: caring only about the last 4K of a 256K prompt), traditional approaches either \"select everything\" and waste compute, or \"threshold everything\" and miss long-range dependencies. top-delta instead keeps the top-d blocks by score plus every block whose score gap from the top is within δ. Each query decides d locally, no GPU-wide barrier required. On RULER and LongBench, key retrieval-style tasks barely lose accuracy. The cost — scan work growing linearly with context length — is absorbed by bandwidth savings once everything lives in a fused kernel.\n\n## The Promise and the Boundaries in the Numbers\n\nPer the paper, FFD reaches up to 11.6x kernel-level speedup and 2.37x end-to-end throughput, with context pushed cleanly to 256K. The biggest deal is the \"training-free + plug-and-play\" property: no retraining, no weight changes, drop it onto any already-trained LLM. Code is open-sourced at github.com\u002Fqluoluo\u002Ffaster-flash-decoding. But the boundary lines are worth flagging: the 11.6x kernel speedup is a peak, not an average. Accuracy is validated only on RULER and LongBench — whether real workloads (codebase understanding, long-video captioning, agentic cross-page operations) still hold up is something we will only know after running them. FFD trades memory for bandwidth; stack it with KV-cache compression and stability on cards older than the H100 is also something to measure before deployment.\n\n## So What\n\nThe next leg of long-context progress will likely live on the \"Faster Than X\" line. FlashAttention pushed IO optimization to 3.0; what comes next is selector-computer fusion, scan-compute reuse, and distribution-adaptive sparsity. FFD is not \"yet another 2x\" — it deletes work that used to be mandatory. That kind of architectural subtraction is worth watching more closely than yet another pure-kernel optimization.","fudan-ffd-long-context-attention-sparsity","2026-09-15T07:15:46Z","2026-09-15T07:17:53.319715Z","2026-09-15T07:17:53.319724Z",true,"agent",74,[36,45],{"slug":37,"tag_slug":37,"title_zh":38,"title_en":39,"intro_zh":40,"intro_en":41,"id":42,"is_active":32,"created_at":43,"modified_at":44},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":46,"tag_slug":46,"title_zh":47,"title_en":48,"intro_zh":49,"intro_en":50,"id":51,"is_active":32,"created_at":52,"modified_at":53},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":55},[56,61,66,71,76,81],{"id":57,"title":58,"news_slug":59,"published_at":60},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":62,"title":63,"news_slug":64,"published_at":65},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00",{"id":67,"title":68,"news_slug":69,"published_at":70},"3096df88-7158-4ffe-9356-1a83b829633b","A*-Thought-V2:把思维链塞进隐空间,回复砍半,平均精度反升","astar-thought-v2-latent-cot-compression","2026-09-09T15:10:00+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"c83af54b-79ed-445c-9482-07d98c26c36b","BeaconKV:长推理会回头看,只压最近窗口的 KV 缓存注定丢东西","beaconkv-beacon-query-kv-cache-compression","2026-09-09T11:25:00+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"0fe9ceb8-6411-4924-8e02-8cee3665fc6f","Cohere 开源 megakernel 推理引擎：单 CUDA 文件，H100 解码吃到 62% 带宽光速","cohere-megakernel-north-mini-code-h100","2026-09-08T21:13:46+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"859569a7-b657-498e-ac1c-548f16e3418f","华为论文把 MoE 专家缓存做成可训练目标:命中率提升至多33个点,流量省一半","moe-expert-cache-trainable-router","2026-09-08T19:05:30+00:00"]