[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-f76e6f4b-1ae5-442a-abfa-823e7226ae81":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"f76e6f4b-1ae5-442a-abfa-823e7226ae81","Hugging Face 把 transformers 跑出 vLLM 原生速度：单 flag 让 235B MoE 直接吃到 EP 红利","7 月 8 日,Hugging Face 在官方博客宣布 transformers 的 vLLM backend 已经跑出和定制化 vLLM 实现持平甚至更快的速度。三档 Qwen3 模型横评结果:4B dense 单卡、32B dense 用 tensor parallelism、235B-A22B-FP8 MoE 在 8×H100 节点上做 data + expert parallel,三种部署形态下 throughput 全部 meet or beat 原生 vLLM 实现。\n\n过去一年,模型作者要在 transformers 之外再为 vLLM 手写一份并行化 kernel——MergedColumnParallelLinear、QKVParallelLinear、MoE 的 EP 融合——才能拿到极限性能。这次的核心思路是用 torch.fx 对模型图做静态分析,找到可优化模式后再通过 ast 直接改写源码,把 fused op 注入到 transformers 模型定义里。换句话说「transformers 写一次,vLLM 自动拿到 native 速度」,作者不再需要双份维护。\n\n更深的影响是训练-推理统一:transformers 模型既能 inference 又能直接跑在 RL rollout \u002F 训练里,过去手写 vLLM 模型只能 inference。配合 --model-impl transformers 一个 flag,Qwen3-4B、Qwen3-32B、Qwen3-235B-A22B-FP8 都能在 vLLM 引擎里跑到原生吞吐。\n\n这是一次生态位重排。HF 把「参考实现」推向了「参考实现 = 生产部署」;对开源社区来说,新模型作者可以专注一份代码同时覆盖训练和推理服务;对 vLLM 来说,原生模型实现的护城河变浅了,但模型库覆盖速度会显著加快。当「transformers-native 速度」成为默认,模型分发和推理部署之间的耦合会被进一步打破。","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fnative-speed-vllm-transformers-backend","24d5c6c5-6573-4180-a1fd-f1459842d1af",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source","2026-07-09T04:03:00Z","2026-07-09T04:07:51.273572Z","2026-07-09T04:07:51.273583Z",true,"agent",3]