DeepSeek V4's technical breakthrough isn't just at the architecture level, but hidden in training infrastructure innovation.
On April 24, DeepSeek released the V4 preview, announcing that the Pro version (1.6T total parameters, 49B activated) and Flash version (284B total parameters, 13B activated) both support 1-million-token context, with API pricing only a fraction of GPT-5.5. More noteworthy is the technical path achieving all this: DeepSeek is the first to use FP4 mixed precision at scale in pretraining — MoE expert weights are stored in FP4, other parameters run in FP8, achieving 9.5 to 13.7× memory reduction compared to V3.2.
FP4 quantization compresses weight precision to 4-bit, saving about 50% VRAM compared to FP8. This strategy isn't simple "brute-force compression" but maintains model quality via quantization-aware training paths. DeepSeek also introduced a new optimizer called Muon, which per the official description can "accelerate convergence and improve training stability" — the first time a major player introduces a brand-new optimizer architecture in super-large-scale training since the Adam series.
V4-Pro was trained on 32T+ tokens. The Codeforces score of 3206, surpassing GPT-5.4's 3168, is the most convincing capability proof. But beyond the capability leap, what truly deserves attention is: extreme low-precision training is moving from "expedient measure" to "standard paradigm." FP4/FP8 mixed precision dramatically lowers the hardware threshold for training ultra-large-parameter MoE models, and lays the foundation for running ultra-large models on domestic chips like Huawei Ascend NPU. DeepSeek V4 has been confirmed runnable on both Nvidia GPUs and Huawei Ascend NPUs, which means the domestic compute ecosystem has taken a substantial step forward.
This isn't just "cheaper," but a fundamental shift in training infrastructure paradigm. Extreme low-precision training is redefining what level of hardware can train what scale of model, and DeepSeek has blazed this trail first.