[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-gemma-4-mtp-drafter-74m-3x-local":3,"topics-all":36,"news-related-98bece1e-3ea5-4ee6-b4b7-b4e9aa385fbc":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"98bece1e-3ea5-4ee6-b4b7-b4e9aa385fbc","Gemma 4为何变快了：Google多Token预测让本地推理提速3倍","Google为Gemma 4开源模型族推出了实验性多Token预测（MTP）drafters，在消费级GPU上实现高达3倍的推理加速。这是自Gemma 4今年春季发布以来最重要的技术迭代，也是Google将旗舰模型优化技术下放到本地推理场景的又一次尝试。\n\n投机解码的精髓：小模型替大模型「打草稿」\n\n传统大语言模型逐token生成，每个token的计算量相同——无论是填充词还是关键推理步骤。这意味着在消费级GPU上运行几十B参数模型时，显存带宽往往成为瓶颈：处理器等待权重数据从显存搬到计算单元的时间，远超实际计算时间。\n\nMTP drafters的思路很巧妙：用一个小模型（仅7400万参数）为大模型提前「猜」出多个后续token，再由主模型并行验证这些猜测。猜对了，直接跳过后者的计算；猜错了，主模型接管正确路径。Gemma 4的E2B和E4B drafters还共享KV Cache——主模型已算出的上下文无需重复计算，加上稀疏解码技术将token候选范围压缩到高概率簇，进一步降低计算开销。\n\n实测数据：在RTX PRO 6000上提速3倍\n\nArs Technica报道了在NVIDIA RTX PRO 6000显卡上的对比测试：标准自回归推理与启用MTP Drafter的Gemma 4 26B相比，输出质量相同，等待时间减半，换算为每秒生成token数提升约3倍。这一数字与Google TPU团队此前公布的投机解码加速数据一致，但这次面向的是本地GPU部署场景。\n\n开源与效率，Google的两条腿\n\nGemma 4本次更新值得关注的还有其定位：Google同步将许可证切换为Apache 2.0，对商业使用完全友好。在Llama、Mistral等开源模型竞争日趋激烈的背景下，推理效率的优化成了差异化的关键战场——模型不仅要跑得动，还要跑得快。MTP drafters的出现表明，Google正在将Gemini系列的前沿优化技术系统性地移植到开源Gemma模型上。\n\n这也呼应了2026年整个LLM行业的主旋律之一：从「模型越大越好」转向「推理越省越好」。无论是投机解码、KV Cache压缩还是量化技术，核心矛盾都是一样的——如何在消费级硬件上榨出更多有效算力。Gemma 4的MTP drafters是这个趋势里最新、最具体的一个案例。\n\n对本地AI玩家而言，Gemma 4 MTP drafters已在Hugging Face上开放，74M参数的小体积意味着树莓派以外的中端设备也有望受益。如果你正在本地运行Gemma 4，不妨更新到最新版本试试——速度的提升可能超出预期。","https:\u002F\u002Farstechnica.com\u002Fai\u002F2026\u002F05\u002Fgoogles-gemma-4-open-ai-models-use-speculative-decoding-to-get-up-to-3x-faster\u002F","4d11edad-2df6-45f6-b71f-70f65de7f7fd",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"8cf7490f-2449-4ba7-be19-61befa0d92b4","google",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"116c439e-62b8-4be9-8341-c899c14d865c","en","Why Gemma 4 is faster: multi-token prediction triples speed","Google has launched experimental Multi-Token Prediction (MTP) drafters for the Gemma 4 open-source model family, achieving up to 3× inference speedup on consumer-grade GPUs. This is the most important technical iteration since Gemma 4's spring release this year, and another attempt by Google to bring flagship-model optimization technology down to local inference scenarios.\n\n**The essence of speculative decoding: small model \"drafts\" for the large model**\n\nTraditional large language models generate token by token, with the same compute amount per token — whether it's a filler word or a key reasoning step. This means that when running tens-of-B-parameter models on consumer-grade GPUs, VRAM bandwidth often becomes the bottleneck: the time the processor waits for weight data to move from VRAM to the compute unit far exceeds the actual compute time.\n\nThe MTP drafters' approach is clever: a small model (only 74 million parameters) \"guesses\" multiple subsequent tokens for the large model in advance, then the main model verifies these guesses in parallel. If correct, skip the latter's computation; if wrong, the main model takes over the correct path. Gemma 4's E2B and E4B drafters also share KV Cache — the main model's already-computed context doesn't need recomputation, plus sparse decoding technology compresses the token candidate range to high-probability clusters, further reducing compute overhead.\n\n**Real-world data: 3× speedup on RTX PRO 6000**\n\nArs Technica reports comparison testing on NVIDIA RTX PRO 6000 graphics cards: standard autoregressive inference vs Gemma 4 26B with MTP Drafter enabled, same output quality, half the wait time, translating to about 3× more tokens generated per second. This number is consistent with the speculative-decoding speedup data Google TPU team previously published, but this time it's targeting local GPU deployment scenarios.\n\n**Open source and efficiency, Google's two legs**\n\nAnother noteworthy aspect of this Gemma 4 update is its positioning: Google simultaneously switched the license to Apache 2.0, fully friendly to commercial use. Against the backdrop of increasingly fierce competition from open-source models like Llama and Mistral, inference efficiency optimization has become a key differentiation battlefield — models not only need to run, but run fast. The emergence of MTP drafters shows Google is systematically porting Gemini series frontier optimization technology to open-source Gemma models.\n\nThis also echoes one of the main themes of the 2026 LLM industry overall: a shift from \"the bigger the model the better\" to \"the cheaper the inference the better.\" Whether speculative decoding, KV Cache compression, or quantization, the core contradiction is the same — how to squeeze more effective compute out of consumer-grade hardware. Gemma 4's MTP drafters is the latest, most concrete case in this trend.\n\nFor local-AI players, Gemma 4 MTP drafters is open on Hugging Face; the 74M-parameter small size means even mid-range devices beyond Raspberry Pi can benefit. If you're running Gemma 4 locally, consider updating to the latest version to try — the speedup may exceed expectations.","gemma-4-mtp-drafter-74m-3x-local","2026-05-06T19:00:00Z","2026-05-06T19:07:03.961979Z","2026-08-19T02:08:40.142862Z",true,"agent",171,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"91f81ab2-f8c9-47a1-8919-3165d03f44b0","Gemma 4 26B：开源MoE模型的性价比新标杆","gemma-4-26b-moe-3-8b-active-256k-apache2","2026-04-26T19:00:00+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"68812025-96eb-4ca9-a1bc-8a82a40174dc","Google RRSI:给 Agent 外壳自进化加正则化","google-rrsi-agent-harness-regularization","2026-09-22T23:08:34+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"3559e613-9558-48e1-ab20-f53b62796363","让每个 token 用上全部专家:高德 IntBMoE 解耦参与度、计算与显存,60ms 服务数亿用户","intbmoe-full-participation-block-moe","2026-09-21T13:01:54+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"28c41f06-d20f-481c-b133-cd109af3aed1","答对之后停不下来:微软团队揪出在线蒸馏的 EOS 错配元凶","eos-mismatch-opd-length-inflation","2026-09-18T21:09:06+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"2e27016d-b90e-45c7-825a-41fd1e435c80","JHU 新研究:组合持续学习机制,百任务记忆留存从 1.2% 提到 34.9%","compose-cl-long-horizon-memorization","2026-09-16T15:10:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"30fca629-bace-4832-9789-b44aa8c8989d","学生团队从零训出开源 7B 模型 ZGCM-1:数学推理硬刚 235B 前沿","zgcm-1-open-7b-foundation-model","2026-09-15T19:10:00+00:00"]