[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-62e17707-e36f-45f6-8749-0d0370382cbd":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"62e17707-e36f-45f6-8749-0d0370382cbd","llm-d 在三厂商混合 GPU 集群上跑出 3-5 倍加速：KV Cache 感知路由让 IBM Granite 和 Sarvam-30B 同时受益","IBM Research、Red Hat 与印度主权云 NxtGen 合作，把开源推理框架 llm-d 部署到由三种不同厂商 GPU 组成的集群中（20 张 pod 跨 A\u002FB\u002FC 三家），并在 2026 年 6 月 23 日公开了实验结果：相比传统 Kubernetes 轮询调度，llm-d 的 KV Cache 感知路由器把峰值吞吐从约 9,600 tokens\u002Fs 拉到 14,200 tokens\u002Fs，高负载下从 7,500 tokens\u002Fs 几乎翻倍，同时把首 token 时间（TTFT）缩短近 30 秒。技术核心是硬件无关的前缀缓存路由：实时追踪每个 vLLM 实例的 KV Cache 状态，把请求送到最可能命中前缀的节点，并显式分离 prefill 与 decode 阶段以便分别优化。算账环节同样亮眼——以 Sarvam-30B 服务 1,000 并发用户为例，按 \u002FGPU·h 计，llm-d 每年可省约 525 万美元，硬件成本几乎可以减半；同一套集群能够服务的用户数也接近翻倍。llm-d 早已捐给 CNCF，本次实验的价值在于第一次系统证明「不同年代、不同厂商的 GPU 可以共存于同一条推理服务线」——便宜的老卡承担低优先级或批处理任务，最新的高端卡专注关键 SLA，企业不必为每次新模型都全套换血。这条路线对正在推进主权云、又想压住推理成本支出的机构格外有借鉴意义，也意味着 vLLM\u002FSGLang 这类引擎正在从「单集群优化器」迈向「异构时代的操作系统」。","https:\u002F\u002Fresearch.ibm.com\u002Fblog\u002Ffast-inference-mixed-gpus","6e1b5ecb-cb95-4c11-9d4e-6e6cd8d11a70",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source","2026-06-23T22:00:00Z","2026-06-23T22:13:41.224949Z","2026-06-23T22:13:41.224960Z",true,"agent",3]