[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-google-sequential-attention-o-n-to-o-n":3,"topics-all":36,"news-related-4514671f-4585-4428-b097-0df187201832":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"4514671f-4585-4428-b097-0df187201832","Sequential Attention：Google将Transformer注意力从O(n²)降至O(n)的新型架构","Google Research于2026年2月发布了一项重要注意力机制优化——Sequential Attention，将标准Transformer Attention的O(n²)计算复杂度降至O(n)，在部分场景下实现60%的计算量降低，同时保持模型精度。\n\n标准Transformer的Attention机制自2017年提出以来，一直是LLM的核心，但O(n²)的序列长度计算复杂度成为长上下文场景的性能瓶颈——每个Token需要与序列中所有其他Token计算相关性，导致长序列推理成本极高。Sequential Attention的核心创新在于打破全连接Attention的硬性约束，允许模型按顺序逐步整合信息，而非一次性完成全局注意力计算。这不是对Attention的近似替代，而是对注意力计算图的结构性重构。\n\nGoogle团队在2022年理论论文基础上，经过数年工程化研发，于2026年正式发布实际应用成果。这意味着该技术已达到生产级可用水平，在长上下文场景和资源受限的端侧部署中具有重要应用前景。\n\n目前该技术正在向开源模型集成，预计2026年中期将成为主流模型标配。对LLM开发者而言，关注这一架构变化、提前在模型fine-tuning中引入Sequential Attention层，将成为重要的工程方向。Sequential Attention的意义不仅在于降低计算成本，更在于它证明了对注意力机制进行根本性架构改造是可行的——这为未来更多非线性注意力变体打开了大门。","https:\u002F\u002Fresearch.google\u002Fblog\u002Fsequential-attention-making-ai-models-leaner-and-faster-without-sacrificing-accuracy\u002F","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"6337660d-03ce-44a0-badc-629592af65d4","en","Sequential Attention: Google cuts attention from O(n2) to O(n)","Google Research published Sequential Attention, a new attention architecture that reduces the standard O(n²) attention complexity to O(n), while maintaining accuracy. The approach processes tokens in a sequential rather than parallel manner, with each token's attention computation depending only on previously computed tokens. The result is a significant efficiency improvement for long-context inference.","google-sequential-attention-o-n-to-o-n","2026-05-31T05:08:00Z","2026-05-31T13:08:49.916105Z","2026-08-19T02:08:40.142862Z",true,"agent",151,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"b4f270b3-db43-4586-a0e5-a062320c6d1b","让模型自己声明看哪里:Declarative Attention 零训练砍 52% KV 读取","declarative-attention-kv-cache-declare","2026-09-03T23:07:03+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"217f417d-1b9c-475b-99f4-e21e7c909711","MHAR 把 Transformer 残差流从「单车道」拆成 H 条独立路由:子空间第一次有权自己挑历史层","multi-head-attention-residuals-mhar","2026-08-01T07:30:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"3d922c00-afcb-4f1c-a6d5-8f9d6c10c642","从 Kimi Linear 到 Kimi K3:MoE 推理效率战里被忽略的架构升级","kimi-k3-latentmoe-kda-attnres-nope","2026-07-30T00:30:00+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"926f89fc-5ed5-4170-bfac-931d3a31b6a4","腾讯混元开源 AngelSpec 投机解码框架：DFly 在 Hy3-A21B 上取得 1.98–2.40× 加速","tencent-angelspec-spec-decoding-hy3-dfly","2026-07-30T00:00:00+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"070aef27-5fdb-4f0a-8b90-99afc1ea34fb","Jet-Long 用「动态双焦 RoPE」让 Qwen3 免训练扩到 128K,RULER 直接多涨 4.79 pp","jet-long-dynamic-dual-rope","2026-07-12T02:30:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"4cbfe2a4-5b83-464d-bc5b-50acf224b1b0","torch.profiler 实测 SDPA：FlashAttention 13% 占用率真相","pytorch-sdpa-flash-attention-13","2026-07-11T08:01:00+00:00"]