[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-star-kv-icml-2026-spotlight":3,"news-related-4f0ccd20-9fbe-4abb-abc5-bce8fe34ba32":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"4f0ccd20-9fbe-4abb-abc5-bce8fe34ba32","STAR-KV：把 KV 缓存压到 1\u002F20，ICML 2026 Spotlight 的\"软阈值\"路线","7 月 1 日，韩国 AI 基础设施公司 **Dnotitia** 联合 UC San Diego VVIP Lab 开源了 **STAR-KV**，并入选 ICML 2026 **Spotlight**（约占已接收论文 8.4%）。在 LLaMA-3.1-8B、128K 上下文、batch size 4 的设定下，KV 缓存会吃下约 81% 的显存——STAR-KV 想正面回答\"长上下文推理的显存墙，是否能被压到不构成瓶颈\"。\n\n## 三个招式串成一条管线\n\nSTAR-KV 全称\"**Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control**\"。技术是把三件常见事打包成一条链路：\n\n1. 低秩近似先把 KV 砍掉 75%；2. 混合精度量化把压缩比推到最高 20×；3. 自研 GPU kernel 减少 attention 的冗余访存。\n\n作者报告：attention 计算最高提速 **6.9×**，生成吞吐最高 **3.1×**，并在多个评测上的精度**高于** KVTC（ICLR 2026、Transform Coding）和 Google 的 TurboQuant。代码已开源，下一步计划集成进 vLLM。\n\n## 真正的差异点是\"软阈值\"\n\nKV 压缩这条线，从 H2O 到 KVTC 再到 TurboQuant 已经卷过好几轮。STAR-KV 的差异点在于**\"软阈值 + 自适应秩\"**：模型按每一层、每一段上下文自动决定低秩近似程度，而不是像 KVTC 那样用一套统一的 Transform Coding 系数硬切。**STAR-KV 把\"压多少\"这件事从手工超参变成可学习控制量**，粒度更细，精度损失更可控。\n\n## 工程化正在接管这场仗\n\nICML 2026 主会场就在首尔，Dnotitia 作为韩国 AI Infra 公司把论文送进 Spotlight，叠加开源 + 计划 vLLM 集成，提示一个转折信号：**长上下文推理这条赛道，已经从\"论文比拼加速比\"进入了\"工程化选型\"阶段**。\n\n对中文社区，比起数字，更值得关注的指标是：**谁先在 vLLM \u002F SGLang 上把端到端长上下文吞吐稳定跑出来，并保持精度不掉**。TurboQuant 与 STAR-KV 先后出现，\"长上下文 = 贵\"的认知可能走到工程落地拐点——前提是这些压缩方法真能撑得住长程 Agent 和 1M token 上下文的生产负载。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.01790","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"401e9206-8bde-463b-93ec-5f2efb6eaa70","en","STAR-KV compresses KV cache 20x with soft thresholds (ICML)","On July 1, Korean AI-infrastructure company Dnotitia, together with UC San Diego's VVIP Lab, open-sourced STAR-KV, which has been accepted as an ICML 2026 Spotlight (roughly 8.4% of accepted papers). On the LLaMA-3.1-8B setup with 128K context and batch size 4, the KV cache eats around 81% of GPU memory — STAR-KV sets out to directly answer: \"Can the long-context inference memory wall be compressed to the point where it's no longer the bottleneck?\"\n\n## Three moves chained into one pipeline\n\nThe full name of STAR-KV is \"Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control.\" The technique packages three common ideas into one pipeline:\n\n1. Low-rank approximation first slashes the KV by 75%; 2. Mixed-precision quantization pushes the compression ratio up to 20×; 3. A custom GPU kernel reduces redundant attention memory accesses.\n\nThe authors report: attention compute speedups up to 6.9×, generation throughput up to 3.1×, and accuracy on multiple evaluations higher than KVTC (ICLR 2026, Transform Coding) and Google's TurboQuant. The code is open-sourced, and integration into vLLM is the next milestone.\n\n## The real differentiator is \"soft thresholding\"\n\nThe KV-compression line — from H2O, to KVTC, to TurboQuant — has already gone through several rounds. STAR-KV's differentiator is \"soft thresholding + adaptive rank\": the model decides the low-rank approximation level per layer and per context segment, instead of hard-cutting with a single set of Transform Coding coefficients as KVTC does. STAR-KV turns \"how much to compress\" from a hand-tuned hyperparameter into a learnable control variable — finer granularity, more controllable accuracy loss.\n\n## Engineering is taking over the field\n\nThe ICML 2026 main conference is in Seoul, and Dnotitia — a Korean AI-infra company — landing the paper in Spotlight, combined with the open-source release and the planned vLLM integration, signals a turning point: the long-context inference track has moved from \"papers racing on speedup ratios\" into the \"engineering-selection\" stage.\n\nFor the Chinese community, beyond the headline numbers, the more meaningful metric to watch is: who first gets end-to-end long-context throughput running stably on vLLM \u002F SGLang while keeping accuracy intact. With TurboQuant and STAR-KV arriving in quick succession, the notion that \"long context = expensive\" may be approaching its engineering inflection point — provided these compression methods can actually hold up under long-horizon agent and 1M-token-context production loads.","star-kv-icml-2026-spotlight","2026-07-02T08:00:00Z","2026-07-02T08:07:25.719595Z","2026-08-19T02:08:40.142862Z",true,"agent",218,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"e428fd02-4e0c-4702-8173-9bbebb02cc31","Lynx:渐进式投机量化让长上下文 LLM 的 KV 缓存传输跑出 1.43× 加速","lynx-progressive-speculative-quantization","2026-07-07T10:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"988bbfb8-672a-4c6c-98f7-3a170b6bd8b3","Macaw 把 LFM2.5 装进 1.5GB:4-bit 端侧 LLM 跑 Mac 控制工具链","macaw-lfm25-15gb-edge-mac-agent","2026-08-24T06:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"ddb7bc6c-6b6e-4797-ab76-d1aeab5a3002","压缩得好≠部署得好:树莓派实测边缘 LLM,LoRA恢复模型100题押97个同答案","edge-llm-compression-raspberry-pi","2026-08-23T13:30:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"d2c56430-f9a7-4844-bfab-a8651279a70c","ResKV 不再把 KV 缓存压缩等同于删词：给被淘汰的信息留一份残差账本","reskv-residual-kv-cache-compression","2026-08-03T10:43:23+00:00"]