Since the birth of the Transformer in 2017, the attention mechanism's compute complexity has always been O(n²) — the longer the sequence, the faster compute cost grows, becoming the fundamental bottleneck for large models moving toward long contexts. When context exceeds 100K tokens, the Prefill phase cost can even multiply by several times.

DeepSeek V3.2 introduces DeepSeek Sparse Attention (DSA), directly tackling this problem. DSA changes token-to-token comparison from computing all connections one by one to only preserving local windows, global tokens, and selective long-range links — the rest of attention weights are zeroed out directly. This isn't random dropping, but letting the model learn which connections are worth keeping.

Technical reports show: under 128K context, Prefill cost drops from about $0.65/million tokens to $0.35, Decode cost from about $2.40 to $0.80, a comprehensive reduction of about 50%. In V3.2 benchmarks, long-context task accuracy is essentially on par with full-attention models. DSA has been open-sourced with V3.2-Exp, and the GitHub community has uploaded systematic comparison experiment code, verifying sparse attention efficiency improvement of 139-302%.

More noteworthy, DeepSeek V4 further engineering-ized sparse attention on the basis of V3.2, achieving Day-0 production-grade support with SGLang and Miles inference frameworks. This means DSA is no longer just an academic result, but has entered real inference production pipelines.

Implication for the industry: as model-capability gaps gradually narrow, inference efficiency is becoming the new competitive axis. DeepSeek has taken the lead in proving that reducing long-context cost doesn't have to come at the cost of quality. This will force the entire industry to re-examine the assumption that long context must use full attention.

So what: if your product involves RAG, long-document analysis, or multi-turn agent memory, it's worth recalculating — not all scenarios need full attention, finding the selective long-range links in your business may be the key to the next round of efficiency optimization.