[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-hf-vllm-transformers-backend":3,"topics-all":36,"news-related-f76e6f4b-1ae5-442a-abfa-823e7226ae81":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"f76e6f4b-1ae5-442a-abfa-823e7226ae81","Hugging Face 把 transformers 跑出 vLLM 原生速度：单 flag 让 235B MoE 直接吃到 EP 红利","7 月 8 日,Hugging Face 在官方博客宣布 transformers 的 vLLM backend 已经跑出和定制化 vLLM 实现持平甚至更快的速度。三档 Qwen3 模型横评结果:4B dense 单卡、32B dense 用 tensor parallelism、235B-A22B-FP8 MoE 在 8×H100 节点上做 data + expert parallel,三种部署形态下 throughput 全部 meet or beat 原生 vLLM 实现。\n\n过去一年,模型作者要在 transformers 之外再为 vLLM 手写一份并行化 kernel——MergedColumnParallelLinear、QKVParallelLinear、MoE 的 EP 融合——才能拿到极限性能。这次的核心思路是用 torch.fx 对模型图做静态分析,找到可优化模式后再通过 ast 直接改写源码,把 fused op 注入到 transformers 模型定义里。换句话说「transformers 写一次,vLLM 自动拿到 native 速度」,作者不再需要双份维护。\n\n更深的影响是训练-推理统一:transformers 模型既能 inference 又能直接跑在 RL rollout \u002F 训练里,过去手写 vLLM 模型只能 inference。配合 --model-impl transformers 一个 flag,Qwen3-4B、Qwen3-32B、Qwen3-235B-A22B-FP8 都能在 vLLM 引擎里跑到原生吞吐。\n\n这是一次生态位重排。HF 把「参考实现」推向了「参考实现 = 生产部署」;对开源社区来说,新模型作者可以专注一份代码同时覆盖训练和推理服务;对 vLLM 来说,原生模型实现的护城河变浅了,但模型库覆盖速度会显著加快。当「transformers-native 速度」成为默认,模型分发和推理部署之间的耦合会被进一步打破。","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fnative-speed-vllm-transformers-backend","24d5c6c5-6573-4180-a1fd-f1459842d1af",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"40d560d3-b9af-49c8-a806-8b98ee0c3876","en","HF transformers hits vLLM-native speed with a single flag","On July 8, Hugging Face announced in an official blog post that the vLLM backend of transformers has reached — or even exceeded — the speed of customized vLLM implementations. A horizontal test of three Qwen3 model scales: 4B dense on a single card, 32B dense using tensor parallelism, and 235B-A22B-FP8 MoE doing data + expert parallel on an 8×H100 node — all three deployment forms meet or beat native vLLM implementation throughput. Over the past year, model authors had to hand-write a parallelization kernel for vLLM outside transformers — MergedColumnParallelLinear, QKVParallelLinear, MoE's EP fusion — to get peak performance. The core idea this time is to use torch.fx to do static analysis of the model graph, find optimizable patterns, and then directly rewrite the source code via ast, injecting fused ops into the transformers model definition. In other words, \"write once in transformers, vLLM automatically gets native speed\", and authors no longer need to maintain two copies. The deeper impact is training-inference unification: transformers models can both infer and directly run in RL rollout \u002F training, while hand-written vLLM models can only infer. With a single `--model-impl transformers` flag, Qwen3-4B, Qwen3-32B, and Qwen3-235B-A22B-FP8 can all run at native throughput in the vLLM engine. This is an ecosystem-position reshuffle. HF is pushing \"reference implementation\" toward \"reference implementation = production deployment\"; for the open-source community, new-model authors can focus on one codebase covering both training and inference service; for vLLM, the moat of the native model implementation has become shallower, but the model library coverage speed will significantly accelerate. When \"transformers-native speed\" becomes the default, the coupling between model distribution and inference deployment will be further broken.","hf-vllm-transformers-backend","2026-07-09T04:03:00Z","2026-07-09T04:07:51.273572Z","2026-08-19T02:08:40.142862Z",true,"agent",226,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"1edd86ea-3eb0-4424-9f57-add15c08d891","HF Hub 接入 SkyPilot：Xet 去重让 20+ 云共享模型数据","hf-skypilot-hf-storage","2026-07-12T06:30:00+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"4d436945-18e9-4d69-a4c8-c1e3e975ab33","MiniMax M3发布：稀疏注意力打通百万token上下文，开源模型编程能力逼近闭源前沿","minimax-m3-sparse-attn-million-token-msa","2026-06-04T01:00:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"86c3465b-27f0-4f8c-83d1-9a8e1fa171c8","Compute Aligned Training：让模型学会协同推理的新训练范式","compute-aligned-training-pass-n-vote-michigan","2026-05-22T13:10:00+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"068bcc7a-d901-4d3d-924c-eefa7ced4467","TokenSpeed 开源推理引擎发布：剑指 Agentic Workloads 的高效推理","tokenspeed-agentic-inference-engine","2026-05-20T19:10:00+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"0565190a-0bcd-492f-934f-0ad2ab32f485","70万参数2.8MB填一张表:Cua开源CUA-S1,单次前向替代23轮LLM","cua-s1-forms-system-one-model","2026-09-20T13:11:48+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"b571067a-9fa8-42bf-9431-98f26ac78e03","伯克利把LLM推理搬进SSD:KV缓存压缩15倍","llm-inference-in-flash-cim-ssd","2026-09-19T21:10:00+00:00"]