[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-e428fd02-4e0c-4702-8173-9bbebb02cc31":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"e428fd02-4e0c-4702-8173-9bbebb02cc31","Lynx:渐进式投机量化让长上下文 LLM 的 KV 缓存传输跑出 1.43× 加速","当大模型服务从单体推理走向「Prefill-Decode 分离」的拆解式架构,KV 缓存的跨网络传输就变成了新的瓶颈——必须等 KV 完整搬完,Decode 端才能开始吐字。2026 年 7 月 2 日挂上 arXiv 的论文 *Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inference* 给出了一个相当工程化的解法:把 KV 缓存按位重要性拆成 Anchor(高有效位)和 Residual(低有效位)两条流,Decode 端拿到 Anchor 就先「投机」开始生成,Residual 在后台继续传输,最后做一次精度校验。结果是:Time-to-First-Token 直追激进的 4-bit 量化,精度却能保住 BF16 水平,比标准 8-bit 量化快 1.43×,在多个模型和工作负载上准确率比 SOTA 提升 5.1%。值得注意的是,作者团队来自 SIGCOMM 圈(本来也是投 SIGCOMM 26),他们用网络视角看待 KV 缓存——「既然不同 bit 对注意力的贡献并不均等,那它为什么必须作为一个不可分割的单元整体传输?」这套「先粗后精、双流并行、事后对齐」的范式,既不是纯算法,也不是纯硬件,而是网络和模型推理的协同设计,值得所有做长上下文部署的工程团队认真读一遍。Lynx 的代码与数据已随论文公开,门槛只在于你需要一套支持双流分发的 serving 框架。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.01831","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization","2026-07-07T10:00:00Z","2026-07-07T02:10:35.646080Z","2026-07-07T02:10:35.646090Z",true,"agent",2]