[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-8a42c9c3-a1c7-40fb-8c75-8ac42977b5af":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"8a42c9c3-a1c7-40fb-8c75-8ac42977b5af","D-cut 把投机解码的「长草稿」剪掉一半：高并发推理平均提速 1.65×、MoE 跑出 3×","投机解码（speculative decoding）一直被认为是「无损加速 LLM 推理」的标准配方，但 D-cut（arXiv:2607.14647）把它从教科书推到了真实部署现场：在高并发批处理场景下，长 draft 反而让验证过程消耗大量算力、把投机解码拖得比自回归还慢，这正是当前 vLLM、TGI、TensorRT-LLM 等推理引擎内部最棘手的尾部开销。D-cut 的关键设计只有两条：第一是跨请求联合剪枝（cross-request pruning），把同一个 mini-batch 里所有正在跑的请求当成一张「草稿接受长度热力图」，按草稿置信度重新分配验证预算，让高接受率的请求多验、低接受率的请求少验，避免为注定被拒的 token 白算；第二是把剪枝深度和目标硬件绑定——GPU 架构、并行策略、张量并行度都会改写「验证一个 token」的代价，D-cut 内置一份 runtime cost model，让深度自动适配到 H100、B200 还是消费级卡。实际效果：在 dense 与 MoE 模型上，并发打满时平均 speedup 由 1.26× 跳到 1.65×；部分 dense 配置下，原投机基线已经输给自回归，D-cut 把加速「救回来」；MoE 场景最高 3.0× 速度，逼近单请求理想态。这是一篇工程味很浓的论文，没有改模型结构，不追求榜单虚名，而是把推理服务里那 10%-30% 的尾部时延稳吃下来。值得指出的是，近两年推理优化已经走过「KV cache 量化、推测解码、稀疏 attention」三波，D-cut 把焦点放在 batch 维度的 budget allocation，意味着 LLM serving 的下一战场不再是单请求极致加速，而是「多请求协同调度」的系统题。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.14647","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"045c011e-e2bb-45ce-bdd6-0c927f8a3b87","token-efficiency","2026-07-18T10:10:00Z","2026-07-18T10:14:09.478652Z","2026-07-18T10:14:09.478663Z",true,"agent",4]