[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-nemotron-3-5-bolt-74b-24gb":3,"topics-all":38,"news-related-18653745-99a2-4929-bf64-ad36d6b551ac":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"18653745-99a2-4929-bf64-ad36d6b551ac","74B塞进24GB:英伟达Bolt押注本地智能体","英伟达开源 Nemotron 3.5 Bolt 74B-A9B 24GB:把 120B 的 Super VL 压到 74B 总参、9B 激活,再用 2.0625 比特量化塞进 24GB 显存,RTX Spark 上跑到 84 TPS,官方称准确率超同显存档的 Qwen3.6 35B 和 Gemma 4 26B。","一张 24GB 显存的显卡,以前跑 30B 级模型都要精打细算,英伟达偏要把 740 亿参数的 MoE 模型塞进去,还给它定了条硬指标:解码速度不低于 80 tokens\u002Fs。这就是刚开源的 Nemotron 3.5 Bolt 74B-A9B 24GB,一个为 RTX\u002FDGX Spark 本地智能体任务打造的混合 Mamba-Transformer 架构模型。它的目标很直白:让本地 Agent 和你其他应用共享一张卡,还得活得好。\n\n## 不从头训,把 120B 压过来\n\nBolt 不是从零训练的小模型,而是从老大哥 Nemotron 3.5 Super VL(120B 总参、12B 激活)压出来的:总参数 120B 降到 74B,激活参数 12B 降到 9B,每个 token 命中的专家数从 22 砍到 9。压缩用的工具是英伟达自研的 Star Elastic——一个按预算条件化的路由搜索框架,自动决定每层留多少路由专家、MoE 中间层多宽,目标就是压进 24GB,还把下游量化 的空间提前算进去。官方实验给出了一个反直觉的结论:在给定显存预算下,把大模型压缩到位,比从头训练一个同尺寸小模型再量化到 4 比特效果更好。压缩之后还追加了一轮后训练,在模拟环境里跑端到端强化学习,把视觉功能和本地 Agent 能力尽量找回来。\n\n## 2 比特量化的赌注\n\n真正激进的是量化策略:占权重绝大多数的 FC1\u002FFC2 专家层被压到 IQ2_XXS 格式,每元素只有 2.0625 比特;Mamba 线性层用 NVFP4;lm_head、词嵌入、注意力和 Latent MoE 权重保住 Q8_0。理由写在明面上——RTX\u002FDGX Spark 上的推理瓶颈是权重带宽,权重位宽直接决定每秒能喂多少参数。量化掉的精度,靠量化感知蒸馏(QAD)补回来。这背后是 ParetoQ、Quest 等研究已经铺过的判断:同等显存下,1-2 比特的大网络比 4 比特的小网络更接近 Pareto 前沿。Bolt 相当于把这个学术结论做成了可以直接下载的检查点。\n\n## 速度和榜单\n\n配套的 LiLiCorr 投机解码 drafter 只有 1.13B 参数,一次轻量前向就给整份草稿联合打分;在 vLLM 里用 7 个草稿 token,greedy 解码的平均接受长度做到 4.68。在 RTX Spark 的 AIPerf SPEED-Bench 上,整套组合跑到 84 TPS。官方同步给出的对比里,同显存档的 Qwen3.6 35B-A3B 和 Gemma 4 26B-A4B 在通用 Agent、网页浏览、终端使用等基准上准确率落后;公开榜单口径下,Bolt 在 PinchBench 排第 1,WebArena 和 BrowseComp 排第 2(对比相近显存占位的模型)。提醒一句:这些都是英伟达自报数字,复测以社区为准。\n\n## 明天还有个 4GB 的\n\n据 IT之家报道,英伟达还准备了面向本地语音助手、交互角色和嵌入式界面的 Bolt E4B 4GB,计划 10 月 23 日推出。从 24GB 到 4GB,英伟达在把「本地 Agent 显存档位」铺成一条产品线,支持 llama.cpp、vLLM、Ollama、Unsloth、LM Studio 全家桶,Hugging Face 上量化版和 BF16 版检查点一起放出了。\n\n## 所以呢\n\nBolt 最值得记的不是榜单名次,而是它把一个路线之争摆上了台面:本地智能体到底要「小而密」,还是「大而稀疏+超低位宽」?英伟达用 74B 塞 24GB 押了后者。如果 2 比特权重真能在日常 Agent 任务上站住,那么显存预算固定的前提下,卡脖子的就不再是「能塞进多少参数」,而是「带宽能喂多快」——权重位宽,正在变成端侧模型的新货币。\n\n参考:英伟达官方技术页 [research.nvidia.com\u002Flabs\u002Fnemotron\u002FNemotron-3.5-Bolt-24GB](https:\u002F\u002Fresearch.nvidia.com\u002Flabs\u002Fnemotron\u002FNemotron-3.5-Bolt-24GB\u002F);IT之家报道 [ithome.com\u002F1\u002F013\u002F679.htm](https:\u002F\u002Fwww.ithome.com\u002F1\u002F013\u002F679.htm)","https:\u002F\u002Fresearch.nvidia.com\u002Flabs\u002Fnemotron\u002FNemotron-3.5-Bolt-24GB\u002F","474eef8c-e0c3-46cf-adee-c089558220f9",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"8dac812d-3839-4abe-a855-5f56ec9515fd","nvidia",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"666a9345-92c8-401e-b696-0c0cd5a0cf7a","en","NVIDIA Stuffs a 74B MoE Into 24GB With Nemotron 3.5 Bolt","NVIDIA open-sources Nemotron 3.5 Bolt 74B-A9B: the 120B Super VL compressed to 74B\u002F9B active, 2.0625-bit quantized to fit 24GB, 84 TPS on RTX Spark.","A graphics card with 24GB of VRAM used to struggle with 30B-class models. NVIDIA just open-sourced a 74B-parameter MoE that fits inside that budget, with a hard floor of 80 tokens per second on decode speed. Nemotron 3.5 Bolt 74B-A9B 24GB is a hybrid Mamba-Transformer built for everyday agentic tasks on RTX and DGX Spark — machines where the model has to coexist with everything else you run.\n\n## Compressed from 120B, not trained from scratch\n\nBolt is not trained from zero: it is compressed from its bigger sibling Nemotron 3.5 Super VL (120B total, 12B active). Total parameters drop from 120B to 74B, active parameters from 12B to 9B, and the expert top-K per token falls from 22 to 9. The tool is Star Elastic, a push-button, router-guided architecture-search framework: a budget-conditioned router picks how many routed experts each layer keeps and how wide the MoE intermediates should be, targeting a sub-24GB deployment footprint with downstream quantization priced in. NVIDIA's experiments land on a counterintuitive conclusion — for a fixed memory budget, compressing a large model beats training a smaller one from scratch and quantizing it to 4 bits. After compression, additional post-training with end-to-end reinforcement learning in simulated environments recovers vision functionality and local-agent capability.\n\n## The 2-bit bet\n\nThe aggressive part is quantization. The FC1\u002FFC2 expert layers — the bulk of the weights — are quantized to IQ2_XXS at 2.0625 bits per element. Mamba linear layers use NVFP4, while the lm_head, token embeddings, attention, and latent-MoE weights stay at Q8_0. The rationale is explicit: inference on RTX\u002FDGX Spark is mostly weight-bandwidth dominated, so weight bit-width directly sets how many parameters per second the chip can feed. The accuracy lost to quantization is bought back with quantization-aware distillation (QAD). Behind this sits the ParetoQ and Quest line of research, which argued that at equal memory, large networks at 1-2 bits are more Pareto-optimal than smaller networks at 4 bits. Bolt turns that academic claim into a downloadable checkpoint.\n\n## Speed and leaderboards\n\nThe bundled LiLiCorr speculative-decoding drafter has only 1.13B parameters and jointly scores token candidates across an entire draft in one lightweight pass; in vLLM with seven drafted tokens it reaches a mean acceptance length of 4.68 under greedy decoding. On RTX Spark, the full stack hits 84 TPS on the AIPerf SPEED-Bench. In NVIDIA's own comparison, the two closest 24GB-class rivals — Qwen3.6 35B-A3B and Gemma 4 26B-A4B — trail on benchmarks for general agentic capability, web browsing, and terminal use. On public leaderboards, NVIDIA reports Bolt at #1 on PinchBench and #2 on both WebArena and BrowseComp among models with a similar memory footprint. Treat these as vendor-reported numbers until independently reproduced.\n\n## A 4GB sibling lands on October 23\n\nPer IT Home, NVIDIA also has Bolt E4B 4GB on the way, aimed at local voice assistants, interactive characters, and embedded interfaces, planned for October 23. From 24GB down to 4GB, NVIDIA is turning local-agent memory tiers into a product line, supported across llama.cpp, vLLM, Ollama, Unsloth, and LM Studio, with both the quantized and BF16 checkpoints released on Hugging Face.\n\n## So what\n\nThe most memorable thing about Bolt is not a leaderboard rank but the strategic question it forces: for local agents, do you want small-and-dense, or huge-and-sparse with ultra-low bit-widths? NVIDIA bet on the latter by stuffing 74B into 24GB. If 2-bit weights hold up on everyday agent workloads, then under a fixed memory budget the binding constraint stops being how many parameters fit — it becomes how fast the bandwidth can feed them. Weight bit-width is turning into the new currency of on-device models.\n\nReferences: NVIDIA technical page (https:\u002F\u002Fresearch.nvidia.com\u002Flabs\u002Fnemotron\u002FNemotron-3.5-Bolt-24GB\u002F); IT Home coverage (https:\u002F\u002Fwww.ithome.com\u002F1\u002F013\u002F679.htm)","nemotron-3-5-bolt-74b-24gb","2026-10-22T13:15:00Z","2026-10-22T13:10:46.921943Z","2026-10-22T13:10:46.921957Z",true,"agent",29,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"de2cceb2-7d39-4a5f-844e-5a3144667f49","Nemotron 3.5 Lightning 开源：30B 总参 3B 激活的混合 MoE，直接用 NVFP4 配方预训练","nemotron-35-lightning-30b-a3b-open-release","2026-08-16T15:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"2ddb354a-06f7-4781-b2d5-bcfd1faf38a5","Dory 1.0:英伟达4B循环模型,指令遵循反超9B","nvidia-dory-1-0-4b-looped-llm","2026-10-20T21:09:38+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"b44fb160-9027-44cd-b355-a66869004abc","Nemotron 3.5 Super VL 开源:120B 总参只激活 12B","nemotron-3-5-super-vl-open-120b-moe","2026-10-20T17:11:56+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"ab1e6bdf-9433-41f0-be4d-cc0f78523ac2","SonicBoom v0.0.3:一个 .so 干掉手工调优","sonicboom-v003-mlir-compiler","2026-10-19T09:03:05+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"bbfcbee4-8d92-4078-bb1e-eaf4d1c1e146","Beam 权重落地:501B MoE 三精度齐发","beam-1-0-weights-open-agent-stack","2026-10-18T21:10:09+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d529114c-bfd7-4efb-9af4-444d3065b397","Step 5 Preview 开源,2.7 美元\u002F百万输出","stepfun-step-5-preview-600b-moe","2026-10-17T21:12:04+00:00"]