[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-lynx-progressive-speculative-quantization":3,"news-related-e428fd02-4e0c-4702-8173-9bbebb02cc31":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"e428fd02-4e0c-4702-8173-9bbebb02cc31","Lynx:渐进式投机量化让长上下文 LLM 的 KV 缓存传输跑出 1.43× 加速","当大模型服务从单体推理走向「Prefill-Decode 分离」的拆解式架构,KV 缓存的跨网络传输就变成了新的瓶颈——必须等 KV 完整搬完,Decode 端才能开始吐字。2026 年 7 月 2 日挂上 arXiv 的论文 *Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inference* 给出了一个相当工程化的解法:把 KV 缓存按位重要性拆成 Anchor(高有效位)和 Residual(低有效位)两条流,Decode 端拿到 Anchor 就先「投机」开始生成,Residual 在后台继续传输,最后做一次精度校验。结果是:Time-to-First-Token 直追激进的 4-bit 量化,精度却能保住 BF16 水平,比标准 8-bit 量化快 1.43×,在多个模型和工作负载上准确率比 SOTA 提升 5.1%。值得注意的是,作者团队来自 SIGCOMM 圈(本来也是投 SIGCOMM 26),他们用网络视角看待 KV 缓存——「既然不同 bit 对注意力的贡献并不均等,那它为什么必须作为一个不可分割的单元整体传输?」这套「先粗后精、双流并行、事后对齐」的范式,既不是纯算法,也不是纯硬件,而是网络和模型推理的协同设计,值得所有做长上下文部署的工程团队认真读一遍。Lynx 的代码与数据已随论文公开,门槛只在于你需要一套支持双流分发的 serving 框架。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.01831","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"af02875e-5fae-486e-a89a-cab745a8270f","en","Lynx: progressive speculative quant speeds KV transfer 1.43x","As large-model serving moves from monolithic inference to a \"Prefill-Decode split\" disassembled architecture, KV cache cross-network transfer becomes a new bottleneck — the Decode end must wait for KV to be fully transferred before starting to output tokens. The paper *Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inference*, posted to arXiv on July 2, 2026, gives a fairly engineering-flavored solution: split the KV cache into Anchor (high-effective bits) and Residual (low-effective bits) streams by bit importance; the Decode end \"speculatively\" starts generating once it gets the Anchor, while the Residual continues to be transmitted in the background, with a final precision reconciliation. The result: Time-to-First-Token chases aggressive 4-bit quantization, but precision holds the BF16 level, 1.43× faster than standard 8-bit quantization, with 5.1% accuracy improvement over SOTA across multiple models and workloads. Noteworthily, the author team comes from the SIGCOMM circle (originally also submitting to SIGCOMM 26), and they look at KV cache from a network perspective — \"since different bits don't contribute equally to attention, why must it be transferred as an indivisible whole?\" This \"coarse-first then fine, dual-stream parallel, post-alignment\" paradigm is neither pure algorithm nor pure hardware, but a co-design of network and model inference, worth a serious read by all engineering teams doing long-context deployment. Lynx's code and data are public with the paper; the only threshold is that you need a serving framework that supports dual-stream distribution.","lynx-progressive-speculative-quantization","2026-07-07T10:00:00Z","2026-07-07T02:10:35.646080Z","2026-08-19T02:08:40.142862Z",true,"agent",140,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"c32d3160-4e07-4128-890f-4e135aac2cce","CompactifAI 把 Llama 3.3 70B 砍到一半:Multiverse 在 Intel Xeon 6 上跑出 1.9 倍吞吐","compactifai-llama-3-3-70b-intel-xeon","2026-07-26T04:00:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"4f0ccd20-9fbe-4abb-abc5-bce8fe34ba32","STAR-KV：把 KV 缓存压到 1\u002F20，ICML 2026 Spotlight 的\"软阈值\"路线","star-kv-icml-2026-spotlight","2026-07-02T08:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"988bbfb8-672a-4c6c-98f7-3a170b6bd8b3","Macaw 把 LFM2.5 装进 1.5GB:4-bit 端侧 LLM 跑 Mac 控制工具链","macaw-lfm25-15gb-edge-mac-agent","2026-08-24T06:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"ddb7bc6c-6b6e-4797-ab76-d1aeab5a3002","压缩得好≠部署得好:树莓派实测边缘 LLM,LoRA恢复模型100题押97个同答案","edge-llm-compression-raspberry-pi","2026-08-23T13:30:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"4147f71b-eaa7-4d39-91cf-c2c572105e7f","FlashPrefill V2:128K 长文本 prefill 提速 47 倍,块稀疏注意力走进生产框架","flashprefill-v2-block-sparse-prefill","2026-08-21T19:10:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"d2c56430-f9a7-4844-bfab-a8651279a70c","ResKV 不再把 KV 缓存压缩等同于删词：给被淘汰的信息留一份残差账本","reskv-residual-kv-cache-compression","2026-08-03T10:43:23+00:00"]