[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-depthweave-kv-1-8-compression":3,"news-related-ea5c9441-3e40-4116-8103-dfc327a54ffc":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"ea5c9441-3e40-4116-8103-dfc327a54ffc","DepthWeave-KV:跨层残差因子化 + token 路由器,64K 上下文 KV 缓存压到 1\u002F8","长上下文 LLM 推理的瓶颈正在从「算得动」转向「放得下」——KV 缓存占据的显存随上下文长度线性增长,是阻碍百万 token 推理的硬卡点。Anna Cordoba 等人在本周公开的 arXiv 论文 DepthWeave-KV(arXiv:2607.06523)没有走「整层一刀切」的常规压缩路径,而是把相邻 Transformer 层的 Key\u002FValue 状态用一组共享的低秩通道基底做因子化,再在 token 维度留出轻量残差。\n\n真正的关键设计是 token 条件路由:对承载指令、承担检索任务的关键 token 分配更高重建秩,普通 token 大幅压缩;同时从 attention 输出端引入无标定在线误差探针,生成过程中实时调整压缩强度,无需重训基模。配套的融合 CUDA 内核把基查表、残差反量化、注意力投影合在一起,降低解码期访存。\n\n在 LongBench、Needle-in-a-Haystack、L-Eval、长篇 QA 与摘要任务上,DepthWeave-KV 在 64K 上下文拿到了 8.3× KV 内存减量和 72.8 tokens\u002Fs 的吞吐,任务质量逼近全量缓存,并优于已有压缩方案。\n\n从「统一预算」走向「逐 token 自适应」是 KV 压缩的正确方向——压缩不是把所有人压扁,而是给关键 token 留出通道。DepthWeave-KV 把这套直觉工程化,为生产级长上下文服务提供了一条可落地的路径。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.06523","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"6c7ddf04-41f3-4bff-b642-f7186c937352","en","DepthWeave-KV compresses 64K cache to one-eighth","The bottleneck of long-context LLM inference is shifting from \"can it compute\" to \"can it fit\" — the memory occupied by the KV cache grows linearly with context length, which is the hard bottleneck preventing million-token inference. Anna Cordoba et al., in the arXiv paper DepthWeave-KV (arXiv:2607.06523) released this week, didn't take the conventional compression path of \"cutting the whole layer uniformly\", but used a set of shared low-rank channel bases to factorize the Key\u002FValue states of adjacent Transformer layers, with a lightweight residual left on the token dimension. The real key design is token-conditional routing: higher reconstruction rank is allocated to the key tokens that carry instructions and undertake retrieval tasks, while ordinary tokens are compressed significantly; simultaneously, a calibration-free online error probe is introduced from the attention output end, dynamically adjusting compression strength during generation without retraining the base model. The companion fused CUDA kernel combines base lookup, residual dequantization, and attention projection, reducing memory access during decoding. On LongBench, Needle-in-a-Haystack, L-Eval, long-form QA and summarization tasks, DepthWeave-KV achieves 8.3× KV memory reduction and 72.8 tokens\u002Fs throughput at 64K context, with task quality close to full cache, and outperforming existing compression schemes. Moving from \"uniform budget\" to \"per-token adaptive\" is the right direction for KV compression — compression isn't flattening everyone, but leaving channels for the key tokens. DepthWeave-KV engineers this intuition, providing a landing path for production-grade long-context services.","depthweave-kv-1-8-compression","2026-07-07T15:00:00Z","2026-07-08T22:15:00.157435Z","2026-08-19T02:08:40.142862Z",true,"agent",141,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"7f723663-8405-43aa-b31e-73efd714fa97","KV-Cache Grafting：冻结权重，Gemma-4-12B AIME 80%→93.3%","byte-exact-kv-cache-grafting","2026-07-17T06:20:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"3af7d9f7-9cb3-43a3-a338-00d716c8053e","JoLT 用 Tucker + JL 残差把 KV 缓存压到 1\u002F3：让长上下文 LLM 推理不再被显存卡脖子","jolt-tucker-jl-kv-cache","2026-07-15T02:18:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"ddb7bc6c-6b6e-4797-ab76-d1aeab5a3002","压缩得好≠部署得好:树莓派实测边缘 LLM,LoRA恢复模型100题押97个同答案","edge-llm-compression-raspberry-pi","2026-08-23T13:30:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"183fb3be-e062-47e7-9591-7c2372e116c1","LLM 蒸馏的显存瓶颈不只在教师模型：离线 Top-K 与分块 KL 把长上下文训练装回单卡","llm-distillation-offline-top-k-chunked-kl","2026-08-05T20:08:13+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"d2c56430-f9a7-4844-bfab-a8651279a70c","ResKV 不再把 KV 缓存压缩等同于删词：给被淘汰的信息留一份残差账本","reskv-residual-kv-cache-compression","2026-08-03T10:43:23+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"bffbd811-83a4-455a-a441-386dde3661c5","多模态 LLM 边缘推理:压缩、MoE 路由与量化「互锁」才是真战场","multimodal-llm-edge-interlock","2026-07-26T07:00:00+00:00"]