[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-ascend-950-10t-moe-pretraining":3,"topics-all":38,"news-related-b413248d-50f0-47f7-bfcd-f0bd3dce29b4":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"b413248d-50f0-47f7-bfcd-f0bd3dce29b4","昇腾950实践披露:DeepSeek训练材料首触10万亿参数","华为CANN社区公开昇腾950超节点万亿MoE预训练参考实践,DeepSeek部分分析550B、1.6T切分策略及向10万亿参数外推的部署权衡,系其公开训练材料首次触及该量级。方案含FSDP\u002FEP\u002FCP并行、显存通信优化与FlexMuon优化器,经V4-Flash训练验证。","10 月 5 日，华为 CANN 社区公开了一份技术文档：《基于昇腾 950 超节点的万亿 MoE 模型预训练系统参考实践》。据 93913、观察者网等多家媒体报道，其中在介绍 DeepSeek 预训练支持时，明确提到基于 PyTorch 原生训练框架分析 550B、1.6T 模型的切分策略，以及向 10T（十万亿）规模外推时的部署权衡——这是目前公开的 DeepSeek 相关训练技术材料中，模型规模明确触及 10 万亿参数量级的一次披露。\n\n先把量级说清楚。DeepSeek-R1 的参数量是 6710 亿，10 万亿约是它的 15 倍；现役 DeepSeek-V4-Pro 总参数 1.6 万亿、激活 49B，V4-Flash 总参 284B、激活 13B；阿里当前旗舰 Qwen 3.8-Max 约 2.4 万亿。也就是说，10T 大约是国产已发布最大模型的四倍以上。\n\n## 这份文档讲了什么\n\n方案基于 PyTorch 原生训练框架和昇腾 950 超节点，结合 DeepSeek-V4-Flash 的训练验证，系统性回答\"万亿参数 MoE 怎么训\"：\n\n- **多层并行策略**：CANN 在超节点内部及节点之间设计了 FSDP（完全分片数据并行）、EP（专家并行）、CP（上下文并行）等多种并行部署方案，并针对模型规模扩大后出现的显存、通信和计算效率问题做优化；\n- **内存四件套**：针对训练过程中的参数、梯度、优化器状态和激活值分别进行内存优化；\n- **新组件**：引入 FlexMuon 分布式优化器，以及大容量记忆表分布式训练方案，提升超大规模训练的效率与稳定性；\n- **场景扩展**：除模型规模外推外，还覆盖超长序列、多模态、高可用训练等场景。\n\n硬件底座是昇腾 950 超节点：以单机柜 64 张 NPU 卡为基本单元，最大支持 8192 卡高速互联——这个规模的集群，通信延迟控制不好，训练效率会大打折扣，并行策略的价值正在这里。\n\n## 和阿里路线图是一枚硬币的两面\n\n9 月云栖大会上，阿里宣布 Qwen 下一代模型将扩展到 5-10 万亿参数（本站已报道）。这次 CANN 文档的看点在于：阿里那批新闻讲的是\"要训多大\"，这份文档讲的是\"怎么训\"——切分策略、并行部署、通信与显存权衡，全是工程细节。\n\n据网易报道转述，DeepSeek 创始人梁文锋此前表达过优先做好小激活参数模型、同时保留向更大规模外推能力的思路。两条路线并不矛盾：MoE 架构下，总参数决定能力上限，激活参数决定推理成本，10T 储备与高性价比小模型可以并行推进。\n\n## 所以呢\n\n10T 短期内不会是\"明天发布的新模型\"，它更像能力天花板的探路工程。但这份文档传递的信号很明确：当模型规模从千亿走向十万亿，决定成败的不再是单点算力峰值，而是训练系统的并行策略、通信效率和内存管理——这些工程层面的\"脏活累活\"，恰恰是最难复制的技术壁垒。国产算力栈能不能撑起下一代的模型规模，答案就藏在这类文档里。\n\n参考报道：[93913：DeepSeek冲向10万亿参数？华为CANN社区披露超大规模MoE训练方案](https:\u002F\u002Fwww.93913.com\u002F125215.html)\n","https:\u002F\u002Fwww.93913.com\u002F125215.html","9f1de219-3ac1-417b-8bea-c74bb1c662bf",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"b52db7e9-7c58-42c3-9536-5132cb2f8f72","deepseek",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"e0d31e94-ce47-4c8f-831c-d3d2926d42f3","hardware",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"15dc8591-29dc-4a20-a131-44107cc9199b","en","Ascend 950 doc: DeepSeek training touches 10T scale","Huawei CANN's Ascend 950 MoE pretraining reference details DeepSeek 550B\u002F1.6T sharding and 10T extrapolation — first such scale in its public training docs.","On October 5, Huawei's CANN community published a technical document: \"Trillion-Parameter MoE Model Pretraining System Reference Practice on the Ascend 950 SuperPoD.\" According to multiple Chinese tech outlets including 93913 and Observer.com, the DeepSeek pretraining section explicitly covers sharding strategies for 550B and 1.6T models, plus deployment tradeoffs when extrapolating toward the 10T (ten-trillion-parameter) scale — reportedly the first time DeepSeek-related public training material has touched that magnitude.\n\nTo put the scale in perspective: DeepSeek-R1 has 671B parameters, so 10T is roughly 15x larger. The current DeepSeek-V4-Pro has 1.6T total parameters (49B active) and V4-Flash has 284B total (13B active), while Alibaba's flagship Qwen 3.8-Max sits at about 2.4T. In other words, 10T is more than four times the largest publicly released Chinese model to date.\n\n## What the document covers\n\nBuilt on a native PyTorch training framework and the Ascend 950 SuperPoD, validated with DeepSeek-V4-Flash training, the reference systematically answers \"how do you actually train a trillion-parameter MoE\":\n\n- **Multi-level parallelism**: CANN provides FSDP (fully sharded data parallel), EP (expert parallel) and CP (context parallel) deployment strategies inside and across supernodes, with optimizations for the memory, communication and compute-efficiency bottlenecks that appear as model size grows;\n- **Memory optimization across the board**: parameters, gradients, optimizer states and activations each get dedicated memory treatment;\n- **New components**: the FlexMuon distributed optimizer and distributed training for large-capacity memory tables, aimed at efficiency and stability at extreme scale;\n- **Beyond scale**: the practice also covers ultra-long sequences, multimodal training and high-availability training scenarios.\n\nThe hardware substrate is the Ascend 950 SuperPoD: a 64-card single rack as the basic unit, scaling up to 8,192 cards in high-speed interconnection. At that cluster size, parallel strategy is where the training efficiency lives or dies.\n\n## Two sides of the same trillion-parameter coin\n\nAt September's Apsara Conference, Alibaba announced its next-gen Qwen would scale to 5-10 trillion parameters (covered here earlier). The interesting contrast: those headlines were about *how big to train*; this CANN document is about *how to train it* — sharding, parallel deployment, communication and memory tradeoffs, all engineering detail.\n\nPer NetEase reporting, DeepSeek founder Liang Wenfeng had earlier expressed a philosophy of prioritizing smaller active-parameter models while preserving the ability to extrapolate toward larger scale. The two routes are not in conflict: in MoE architectures, total parameters set the capability ceiling while active parameters set inference cost — a 10T reserve and a cost-efficient small model can advance in parallel.\n\n## So what\n\nA 10T model will not ship tomorrow; this reads more like an exploratory engineering reserve. But the signal is clear: as models grow from hundreds of billions toward tens of trillions of parameters, what decides success is no longer peak single-chip compute — it is the training system's parallel strategy, communication efficiency and memory management. That unglamorous systems engineering is precisely the hardest moat to copy. Whether the domestic compute stack can carry the next generation of model scale is a question answered in documents like this one.\n\nReference: [93913: DeepSeek heading for 10 trillion parameters? Huawei CANN community discloses ultra-large-scale MoE training practice](https:\u002F\u002Fwww.93913.com\u002F125215.html)\n","ascend-950-10t-moe-pretraining","2026-10-08T17:08:18Z","2026-10-08T17:08:28.598338Z","2026-10-08T17:08:28.598353Z",true,"agent",572,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"b6d8130b-f989-46a5-baa3-d81e66265ef8","Reflection开源Beam:501B MoE的RL账本","reflection-beam-501b-moe-rl-campaign","2026-11-20T13:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"f02185e7-5f4c-46c6-bb18-575fbf3b1c0a","摩根大通实测:五项前沿配方,四项真有用","jpmorgan-factorial-retest-llm-ingredient-gains","2026-11-19T17:25:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"e8f13686-3f2f-468a-b974-55634bb30795","Metal Sandbox:Mac 集群的 GPU 调度术","metal-sandbox-mac-cluster-gpu-scheduling","2026-11-15T03:15:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"a79651db-8451-4ee7-9091-809dff098c19","英语只占41%:Translated开源多语MoE小模型","lara-mini-base-multilingual-moe","2026-11-14T19:30:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"1c9b3e02-69f2-452f-9137-ff9c2ddcf7fb","手机上的模型工厂:小模型自动定制,五项反超前沿模型","mobile-ai-factory-on-device-slm","2026-11-12T23:15:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"25191dae-d51d-4fdd-a23c-3f1a46563c8a","MoE剪枝别看专家看神经元:留一半,显存省36-43%","moe-neuron-specialization-rampr","2026-11-11T15:30:00+00:00"]