[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-vllm-0-28-kimi-k3-optimization":3,"news-related-ff3b7c9f-e338-4af9-a562-4fda8b1bc929":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"ff3b7c9f-e338-4af9-a562-4fda8b1bc929","vLLM 0.28 发布:584 项提交押注 Kimi K3,每卡省 17GB 显存","开源推理引擎 vLLM 发布 v0.28.0,584 项提交中最大主题是 Kimi-K3 全栈性能优化:共享专家分片每卡省约 17 GiB 显存,自适应投机预算让 DSpark 首字延迟改善约 60%;DeepSeek V4 稀疏 MLA 端到端跑通,KV 缓存新增磁盘卸载。","开源推理引擎 vLLM 发布了 v0.28.0 版本(2026 年 8 月 26 日),这个包含 584 项提交、270 位贡献者(其中 76 位是新面孔)的版本,最大的主题只有一个:让 Kimi-K3 跑得更便宜。作为部署最广泛的开源 LLM 服务引擎之一,vLLM 每个版本的取舍,基本就是推理侧真实需求的一份实时快照。\n\n## Kimi-K3 全栈优化:省钱是主线\n\n发布说明里,Kimi-K3 的性能推进占了最大篇幅,而且是跨层的组合拳:\n\n- **Decode Context Parallel(DCP)**:解码阶段的上下文并行,把长上下文解码负载摊到多卡;\n- **融合 FlashKDA decode\u002Fprefill 内核**、**SiTU 激活支持 MegaMoE**、**GEMM-RS 序列并行**;\n- **合并 all-gather 通信**:内核级加速 1.5~3 倍;\n- **自适应投机 token 预算**:DSpark 场景下首字延迟(TTFT)改善约 60%;\n- **可选共享专家分片**:每张 GPU 省下约 17 GiB 显存;\n- Kimi-K3 还能在 ROCm 上通过 V2 model runner 运行。\n\n这组数字都出自官方 release notes,是工程口径而非营销口径:每卡省 17 GiB,意味着同一批卡能塞进更多副本或更长上下文;TTFT 改善则直接映射到终端用户的等待体感。\n\n## DeepSeek V4 与其他亮点\n\nDeepSeek V4 侧,稀疏 MLA 端到端跑通(覆盖普通解码、MTP、DSpark 投机解码),AMD Quark NVFP4 量化、gfx11\u002Fgfx950 的 ROCm 支持一并落地。基础设施层面:KV 缓存新增磁盘卸载档位,E\u002FP\u002FD 分离式部署进入 Model Runner V2,批处理默认 token 上限从 8192 提到 16384。新接入的模型包括 Muse Glimmer、Ling 3.0 Flash(含 FP8 与混合 MXFP4 专家变体)、Dots3 NOTE、Interns2mobius。破坏性变更要注意:bitsandbytes 迁出为独立插件,Transformers 依赖升到 5.15.0。\n\n## 为什么值得留意\n\nvLLM 是大多数自托管团队的现实默认选项,它的优化优先级就是社区真实部署需求的排序。Kimi K3 和 DeepSeek V4 这两个来自中国团队的开源模型占据一个版本的头两条,说明它们已经跨过\"发布\"阶段、进入大规模实际服务——开源模型的竞争,正在从发布会转移到每 token 成本。(完整改动见 [GitHub release notes](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm\u002Freleases\u002Ftag\u002Fv0.28.0))\n\n所以,如果你在自托管开源模型,这个版本值得直接升;如果还在观望中国开源模型的落地密度,vLLM 的提交记录比任何发布会都诚实。","https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm\u002Freleases\u002Ftag\u002Fv0.28.0","c59a5c6f-3302-4a86-967e-b44417a05850",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"36be0af1-b747-4ce5-b52a-76747f8f7342","en","vLLM 0.28 bets the stack on Kimi K3: 17 GiB saved per GPU","vLLM 0.28.0: 584 commits headlined by a Kimi-K3 push - ~17 GiB saved per GPU, ~60% better DSpark TTFT, DeepSeek V4 sparse MLA, disk KV offload.","The open-source inference engine vLLM has shipped v0.28.0 (August 26, 2026), and with 584 commits from 270 contributors - 76 of them first-time - the release has one dominant theme: making Kimi-K3 cheaper to run. As one of the most widely deployed open LLM serving engines, vLLM's priorities read like a live snapshot of what self-hosting teams actually need.\n\n## The Kimi-K3 full-stack push\n\nThe release notes dedicate their longest section to Kimi-K3, and it is a cross-layer effort:\n\n- **Decode Context Parallel (DCP)** spreads decode-time long-context load across GPUs;\n- **Fused FlashKDA decode\u002Fprefill kernels**, **SiTU activation support for MegaMoE**, and **GEMM-RS** for sequence parallelism;\n- **Combined all-gathers** with 1.5-3x kernel-level speedup;\n- An **adaptive speculative token budget** delivering ~60% better DSpark TTFT;\n- **Optional shared-expert sharding** saving ~17 GiB of memory per GPU;\n- Kimi-K3 now also runs on ROCm via the V2 model runner.\n\nThese numbers come from the official release notes - engineering figures, not marketing. Seventeen GiB per GPU means more replicas or longer contexts on the same hardware, and the TTFT improvement maps directly to how long end users wait.\n\n## DeepSeek V4 and everything else\n\nOn the DeepSeek V4 side, sparse MLA now works end-to-end for plain decode, MTP, and DSpark speculative decoding, joined by AMD Quark NVFP4 support and ROCm enablement on gfx11 and gfx950. Infrastructure-wise: tiered KV cache gained disk offloading, E\u002FP\u002FD disaggregation landed in Model Runner V2, and the default max_num_batched_tokens rose from 8192 to 16384. Newly supported models include Muse Glimmer, Ling 3.0 Flash (with FP8 and hybrid MXFP4 expert variants), Dots3 NOTE, and Interns2mobius. Breaking changes to watch: bitsandbytes moved to an out-of-tree plugin, and Transformers bumped to 5.15.0.\n\n## Why it matters\n\nvLLM is the practical default for most self-hosting teams, so its optimization priorities are a ranking of real deployment demand. That Kimi K3 and DeepSeek V4 - two open-weight models from Chinese teams - take the top two spots in a single release says they are past the announcement stage and into large-scale serving. The open-source model race is shifting from launch events to cost per token. (Full details in the [GitHub release notes](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm\u002Freleases\u002Ftag\u002Fv0.28.0))\n\nSo if you self-host open models, this version is worth upgrading to directly; and if you are gauging how deeply Chinese open-weight models have landed, vLLM's commit log is more honest than any launch event.","vllm-0-28-kimi-k3-optimization","2026-08-29T19:05:00Z","2026-08-29T19:08:57.320117Z","2026-08-29T19:08:57.320127Z",true,"agent",184,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"199cd4ef-f092-45a5-8635-91778dd2bce2","编译即训练：一句规约炼出 83.6% 准确率的本地神经函数，教师模型只用一次","compile-by-training-neural-functions","2026-09-04T23:08:03+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"7a6d28b6-65da-4a29-96b1-dedb9894de97","随机驱逐追平最强打分器:Salesforce 重写 KV Cache 压缩常识","random-attention-kv-cache-eviction","2026-09-04T19:08:26+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"44a035c8-b8a3-48e5-af4f-c76323dac7b5","RWKV7-G1j 13.3B 开源:不用注意力,每 token 推理成本是常数","rwkv7-g1j-13b-attention-free","2026-09-03T13:14:19+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"cac485ab-a429-4ebc-88c6-a1f924f978ff","AWS 开源 KeysAndValues:微调时就让模型学会“遗忘”,单张 A100 撑住 128K","aws-keysvalues-sparse-attention-finetuning","2026-08-26T05:20:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"c94bdf86-5de9-49fe-8c98-0f5c47611bfe","SGLang v0.5.18 发布:大模型冷启动提速 2.38 倍,710 个 PR 都改了什么","sglang-v0-5-18-cold-start-2-38x","2026-08-24T23:15:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"92433e6b-113a-4ada-af77-fbb8995a9850","LFM2.5-DSpark 开源:300M 草稿模型让端侧推理快 2.87 倍,输出零损耗","lfm2-5-dspark-draft-models","2026-08-21T21:10:00+00:00"]