[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-deepseek-v3-2-dsa-sparse-attention-50pct-cost-cut":3,"topics-all":36,"news-related-e4fd45e9-e0fd-4839-973e-909a442ce5ff":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"e4fd45e9-e0fd-4839-973e-909a442ce5ff","DeepSeek V3.2稀疏注意力：如何将长上下文推理成本砍半","自 2017 年 Transformer 诞生，注意力机制的计算复杂度始终是 O(n²)——序列越长，计算成本增长越快，成为大模型迈向长上下文的根本瓶颈。当上下文超过 10 万 token，Prefill 阶段费用甚至可以翻上数倍。\n\nDeepSeek V3.2 引入了 DeepSeek Sparse Attention（DSA），正面冲击这一问题。DSA 将 token 之间的比较从逐一计算所有连接改为只保留局部窗口、全局 token 和选择性长距离链接——其余注意力权重直接置零。这不是随机丢弃，而是让模型自己学习哪些连接值得保留。\n\n技术报告显示：128K 上下文下，Prefill 成本从约 0.65 美元\u002F百万 token 降至 0.35 美元，Decode 成本从约 2.40 美元降至 0.80 美元，综合降幅约 50%。在 V3.2 基准测试中，长上下文任务准确率与全注意力模型基本持平。DSA 已随 V3.2-Exp 开源，GitHub 社区也上传了系统性对比实验代码，验证稀疏注意力效率提升达 139-302%。\n\n更值得关注的是，DeepSeek V4 在 V3.2 基础上将稀疏注意力进一步工程化，配合 SGLang 和 Miles 推理框架实现了 Day-0 生产级支持。这意味着 DSA 已不只是学术成果，而是进入了真实推理生产管线。\n\n对行业的意味：当各厂商模型能力差距逐渐收窄，推理效率正在成为新的竞争维度。DeepSeek 率先证明，降低长上下文成本不必以牺牲质量为代价。这会迫使整个行业重新审视长上下文是否必须用全注意力这一假设。\n\n所以呢：如果你的产品涉及 RAG、长文档分析或多轮 Agent 记忆，值得重新算一笔账——不是所有场景都需要全量注意力，找到业务中的选择性长距离链接，可能是下一步效率优化的关键所在。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2512.02556","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"0af868a9-b98b-4cb1-bfbc-d10d499cbc20","en","DeepSeek V3.2 sparse attention halves long-context cost","Since the birth of the Transformer in 2017, the attention mechanism's compute complexity has always been O(n²) — the longer the sequence, the faster compute cost grows, becoming the fundamental bottleneck for large models moving toward long contexts. When context exceeds 100K tokens, the Prefill phase cost can even multiply by several times.\n\nDeepSeek V3.2 introduces DeepSeek Sparse Attention (DSA), directly tackling this problem. DSA changes token-to-token comparison from computing all connections one by one to only preserving local windows, global tokens, and selective long-range links — the rest of attention weights are zeroed out directly. This isn't random dropping, but letting the model learn which connections are worth keeping.\n\nTechnical reports show: under 128K context, Prefill cost drops from about $0.65\u002Fmillion tokens to $0.35, Decode cost from about $2.40 to $0.80, a comprehensive reduction of about 50%. In V3.2 benchmarks, long-context task accuracy is essentially on par with full-attention models. DSA has been open-sourced with V3.2-Exp, and the GitHub community has uploaded systematic comparison experiment code, verifying sparse attention efficiency improvement of 139-302%.\n\nMore noteworthy, DeepSeek V4 further engineering-ized sparse attention on the basis of V3.2, achieving Day-0 production-grade support with SGLang and Miles inference frameworks. This means DSA is no longer just an academic result, but has entered real inference production pipelines.\n\nImplication for the industry: as model-capability gaps gradually narrow, inference efficiency is becoming the new competitive axis. DeepSeek has taken the lead in proving that reducing long-context cost doesn't have to come at the cost of quality. This will force the entire industry to re-examine the assumption that long context must use full attention.\n\nSo what: if your product involves RAG, long-document analysis, or multi-turn agent memory, it's worth recalculating — not all scenarios need full attention, finding the selective long-range links in your business may be the key to the next round of efficiency optimization.","deepseek-v3-2-dsa-sparse-attention-50pct-cost-cut","2026-05-01T10:15:00Z","2026-05-01T10:09:00.341169Z","2026-08-19T02:08:40.142862Z",true,"agent",252,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"748a4486-34e7-4215-b515-7eb56b3258c5","TwELL：Sakana AI与NVIDIA联合提出稀疏LLM推理加速20%，解决GPU批处理落地难题","sakana-nvidia-twell-20pct-sparse-batch-gemm","2026-05-30T08:20:00+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"2fa66657-afbb-4f03-849a-f420f42cf2ab","Prompt Caching：LLM推理成本削减90%的隐藏利器","prompt-caching-90pct-token-cost","2026-05-26T01:10:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"73f4d31e-745a-4bba-8a0f-38e7564966de","Sakana AI 提出 99% 稀疏性Transformer：在前馈层动刀革新LLM效率","sakana-99pct-sparse-ffn-transformer","2026-05-16T19:04:00+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"137ce22e-389d-47fb-8219-42ca53d6e916","Qwen 3.6 27B 重磅更新：MTP 技术让本地推理提速 2.5 倍","qwen-3-6-27b-mtp-local-2-5x","2026-05-16T01:01:00+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"4556940e-6456-43ce-b9b4-a0a7fa7a5865","MIT 新方法：自适应草稿模型将推理 LLM 训练速度提升 2-3 倍","mit-adaptive-draft-speculative-train-2-3x","2026-05-15T02:05:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"810251ee-b8bf-4fef-a9d8-e167c22ae4c5","BoostLoRA：梯度增强让低秩适配器「自我进化」，小参数也能有大表达","boostlora-gradient-boosting-lora-residual","2026-05-01T05:10:00+00:00"]