[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-deepseek-v4-muon-optimizer-fp4-mixed-precision":3,"topics-all":36,"news-related-98312e5c-0687-41f5-a5b3-596a45563e67":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"98312e5c-0687-41f5-a5b3-596a45563e67","DeepSeek V4训练内幕：Muon优化器与FP4混合精度重塑大模型效率","DeepSeek V4的技术突破不仅体现在架构层面，更藏在训练基础设施的革新里。\n\n4月24日，DeepSeek发布V4预览版，宣布Pro版本（1.6T总参数、49B激活）和Flash版本（284B总参数、13B激活）均支持100万token上下文，API价格仅为GPT-5.5的几分之一。更值得关注的是实现这一切的技术路径：DeepSeek首次在预训练中大规模采用FP4混合精度——MoE专家权重以FP4存储，其余参数以FP8运行，相比V3.2实现了9.5至13.7倍的内存降低。\n\nFP4量化将权重精度压缩到4-bit，相比FP8再节省约50%显存。这一策略并非简单的「粗暴压缩」，而是通过量化感知训练路径保持模型质量。DeepSeek同时引入了名为Muon的新优化器，据官方描述其能「加速收敛并提升训练稳定性」——这是继Adam系列之后，大厂首次在超大规模训练中引入全新的优化器架构。\n\nV4-Pro在32T+ tokens上完成训练。Codeforces评分3206超越了GPT-5.4的3168，是最有说服力的能力证明。但在能力跃升之外，真正值得关注的信号是：极低精度训练正在从「权宜之计」变为「标准范式」。FP4\u002FFP8混合精度让训练超大参数MoE模型的硬件门槛大幅下降，也为华为昇腾NPU等国产芯片运行超大规模模型奠定了基础。DeepSeek V4已确认可在Nvidia GPU和华为昇腾NPU上运行，这意味着国产算力生态向前迈出了实质性一步。\n\n这不只是「更便宜」，而是训练基础设施范式的一次根本性转变。极低精度训练正在重新定义什么级别的硬件可以训练什么规模的模型，而DeepSeek率先把这条路径走通了。","https:\u002F\u002Fapi-docs.deepseek.com\u002Fnews\u002Fnews260424","4194681c-1a38-405d-a917-40e1dc2622ea",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"2ccc30e7-6830-4782-93a5-5ec5a4f759a2","en","Inside DeepSeek V4 training: Muon optimizer and FP4 mix","DeepSeek V4's technical breakthrough isn't just at the architecture level, but hidden in training infrastructure innovation.\n\nOn April 24, DeepSeek released the V4 preview, announcing that the Pro version (1.6T total parameters, 49B activated) and Flash version (284B total parameters, 13B activated) both support 1-million-token context, with API pricing only a fraction of GPT-5.5. More noteworthy is the technical path achieving all this: DeepSeek is the first to use FP4 mixed precision at scale in pretraining — MoE expert weights are stored in FP4, other parameters run in FP8, achieving 9.5 to 13.7× memory reduction compared to V3.2.\n\nFP4 quantization compresses weight precision to 4-bit, saving about 50% VRAM compared to FP8. This strategy isn't simple \"brute-force compression\" but maintains model quality via quantization-aware training paths. DeepSeek also introduced a new optimizer called Muon, which per the official description can \"accelerate convergence and improve training stability\" — the first time a major player introduces a brand-new optimizer architecture in super-large-scale training since the Adam series.\n\nV4-Pro was trained on 32T+ tokens. The Codeforces score of 3206, surpassing GPT-5.4's 3168, is the most convincing capability proof. But beyond the capability leap, what truly deserves attention is: extreme low-precision training is moving from \"expedient measure\" to \"standard paradigm.\" FP4\u002FFP8 mixed precision dramatically lowers the hardware threshold for training ultra-large-parameter MoE models, and lays the foundation for running ultra-large models on domestic chips like Huawei Ascend NPU. DeepSeek V4 has been confirmed runnable on both Nvidia GPUs and Huawei Ascend NPUs, which means the domestic compute ecosystem has taken a substantial step forward.\n\nThis isn't just \"cheaper,\" but a fundamental shift in training infrastructure paradigm. Extreme low-precision training is redefining what level of hardware can train what scale of model, and DeepSeek has blazed this trail first.","deepseek-v4-muon-optimizer-fp4-mixed-precision","2026-04-28T01:01:00Z","2026-04-28T01:10:16.051586Z","2026-08-19T02:08:40.142862Z",true,"agent",533,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"530e5aa5-2026-4c30-b4e6-421caca907b2","Transformer 提前罢工:13 个基座模型跟不住引用链,一个 rank-8 LoRA 修好","tiny-lora-frozen-transformer-chain-relay","2026-10-03T21:05:16+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"88298830-f6d5-4a61-ad94-27daaa568a9a","Qwen 3.5 新一代 MoE 架构解读：Shared Expert + 混合注意力重塑长上下文效率","qwen-3-5-shared-expert-hybrid-attention-amd","2026-05-05T07:05:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"3a419e1c-fb2b-467e-a5b6-a7188525bc03","TraceForge:推理痕迹自己造,学生反超教师真迹","traceforge-family-native-distillation","2026-10-22T15:10:42+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"fd29c2f1-175f-416e-bde2-95dd65ca3465","KronLSTM 精准改写记忆:线性注意力长上下文新解","kronlstm-kronecker-delta-rule","2026-10-20T15:06:16+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"559df680-f13e-48f0-9263-73116aaff411","上交大DLite免历史投机解码:Agent提速2.74倍","dlite-history-free-speculative-decoding","2026-10-19T17:07:52+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"ab1e6bdf-9433-41f0-be4d-cc0f78523ac2","SonicBoom v0.0.3:一个 .so 干掉手工调优","sonicboom-v003-mlir-compiler","2026-10-19T09:03:05+00:00"]