[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-mosaickv-mosaic-compression":3,"news-related-dd9d199c-5cd9-4a15-8ae6-7e4fb40f4129":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"dd9d199c-5cd9-4a15-8ae6-7e4fb40f4129","MosaicKV:把 KV 缓存压成「马赛克」,长上下文推理跑出 16× 注意力加速","把 KV Cache 当瓶颈来压,是百万级上下文 LLM 服务的标准动作。但多数工作只在序列或通道单轴压缩——再多压,精度就断崖式下滑。\\n\\n7 月 1 日挂上 arXiv 的 MosaicKV (2607.00760) 把这件事推到二维:先识别每个 KV 向量里真正重要的位置,把缓存切成多段,对各段施以不同压缩策略,再用闲置 GPU\u002FCPU 维护压缩态,把注意力从压缩缓存里直接算出来。\\n\\nH800 上结果很硬:attention 加速最多 16×、decode 延迟降 4.8×、吞吐升 7.3×,内存降到 1\u002F3,LongBench\u002FRULER 平均精度只掉 1.76%。\\n\\n三个值得留意的点:2D 压缩的关键不是\"压得更狠\",而是承认 KV 内部本就稀疏不均——延续了 STAR-KV、InfoKV 的\"软阈值\"路线;压缩管理与算力调度捆绑,意味着对框架依赖不轻,落地大概率走 SGLang\u002FvLLM 集成;1.76% 精度换 4–7× 吞吐,百万上下文 Agent 的边际收益正以\"长尾任务终于跑得起\"兑现。\\n\\nMosaicKV 不会让\"上下文无限长\"一夜实现,但把 Agent 时代\"长 prompt 跑不动\"再往生产推了一步。精度损失压到 1%、吞吐翻 5 倍以上,长上下文 LLM 的服务定价和本地推理能力,都会有一轮悄悄的重排。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.00760","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"ad275f62-d5b3-4d66-83d4-a65227f016a0","en","MosaicKV tiles the KV cache for 16x attention speedup","Treating KV Cache as a bottleneck to compress is the standard move for million-token context LLM services. But most work only compresses on a single axis of sequence or channel — compress more, and accuracy drops off a cliff. MosaicKV (2607.00760), posted to arXiv on July 1, pushes this to two dimensions: first identify the truly important positions within each KV vector, cut the cache into segments, apply different compression strategies to each segment, then use idle GPU\u002FCPU to maintain the compressed state, and compute attention directly from the compressed cache. On H800, the results are hard: attention speedup up to 16×, decode latency down 4.8×, throughput up 7.3×, memory down to 1\u002F3, with only 1.76% average accuracy drop on LongBench\u002FRULER. Three points worth noting: the key to 2D compression isn't \"compress harder\", but acknowledging that the KV interior is inherently unevenly sparse — continuing the \"soft threshold\" route of STAR-KV and InfoKV; compression management is bound to compute scheduling, meaning no light dependence on the framework, landing most likely through SGLang\u002FvLLM integration; 1.76% accuracy for 4-7× throughput, the marginal benefit of million-context Agents is being realized as \"long-tail tasks finally runnable\". MosaicKV won't make \"infinite context length\" a reality overnight, but pushes \"long prompt can't run\" in the Agent era one more step toward production. With accuracy loss compressed to 1%, throughput quintupled, the service pricing of long-context LLM and local inference capability will have a quiet reshuffle.","mosaickv-mosaic-compression","2026-07-03T18:01:00Z","2026-07-03T18:08:17.896582Z","2026-08-19T02:08:40.142862Z",true,"agent",199,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"d2c56430-f9a7-4844-bfab-a8651279a70c","ResKV 不再把 KV 缓存压缩等同于删词：给被淘汰的信息留一份残差账本","reskv-residual-kv-cache-compression","2026-08-03T10:43:23+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"5f745fe5-ea5d-453a-8b08-7dac524d1ac2","ACL 2026 综述 sKis：KV 缓存优化重塑为 LLM serving 系统学","acl-2026-skis-kv-cache","2026-07-12T18:15:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"c1ee21bd-4b59-418a-9b77-e46c790a8978","InfoKV 把 KV 缓存压缩推过「只看注意力」的临界点：用信息熵帮推理模型跑得更长","infokv-entropy-kv-cache-compression","2026-06-27T18:14:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"c4380194-5bf9-43a0-8460-46436a4f2f97","LCLM 把上下文压到 1\u002F16：8.8 倍提速的代价是 16 倍时准确率只剩 75%","lclm-1-16-compress-8-8x-75pct-accuracy","2026-06-15T04:00:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"94e7739c-f218-4dfc-803d-3662a97321f3","DeepSeek V4 混合注意力架构解析：如何在1M上下文下将计算量降至原来的27%？","deepseek-v4-csa-hca-1m-27pct-flops","2026-05-27T07:20:00+00:00"]