arXiv 2606.26875 introduces InfoKV, a KV cache compression method that breaks through the "only look at attention" paradigm. The core idea: directly use information entropy to measure each KV entry's "actual contribution to generation," and decide what to evict based on entropy rather than attention weight.
Traditional KV cache compression is mostly attention-based: keep the high-attention entries, evict the low-attention ones. The flaw: attention weight is not the same as "actual contribution to generation." A low-attention entry may still carry critical contextual information; a high-attention entry may be redundant. InfoKV uses a per-entry information-entropy estimation, combining attention weight with a "decay-of-impact-on-output-probability" measure, to make a more accurate keep-or-evict decision.
Technical details: InfoKV uses a small auxiliary network to predict the "delta in output probability" of each KV entry, and uses the entropy of that delta as the keep/evict signal. The auxiliary network is trained end-to-end with the main model, adding no extra inference cost.
Experimental results: at the same memory budget, InfoKV retains 18-25% more generation quality than attention-only compression, with particularly notable gains on long-chain reasoning and code-generation tasks. At the same quality target, InfoKV saves 30-40% more KV cache memory.
The bigger signal: InfoKV is pushing KV cache compression from "engineering heuristics" to "information-theoretic principles." The "attention weight = importance" assumption has dominated for years, and InfoKV may be the beginning of a wave of "information-entropy-driven" compression. For the industry, this means reasoning models can run longer context at the same memory cost — a direct release of the "long reasoning" productivity ceiling.