Huawei open-sourced Pangu 2.0, the next-generation Pangu LLM. The standout: 505B-parameter MoE with 6B active per token, the largest open-source model from a Chinese tech giant. The release is a major step in Huawei's "full-stack open source" strategy — the model, the training code, the data pipeline, and the inference framework are all open-sourced.
The "505B / 6B active" architecture: Pangu 2.0 uses a Mixture-of-Experts (MoE) architecture with 505B total parameters and 6B active per token. The "low active" ratio (1.2%) is unusual — most MoE models have 5-10% active. The "low active" design gives very efficient inference (close to a 6B model) while maintaining the capacity of a 505B model.
The "full-stack open source" highlight: the release includes (1) the Pangu 2.0 weights; (2) the training code (including the data preprocessing, RLHF, and evaluation); (3) the MindSpore inference framework, optimized for Huawei's Ascend chips; (4) the "Ascend NPU" deployment guide. This is the most complete open-source release from a Chinese tech company.
The benchmark: on MMLU, HumanEval, and GSM8k, Pangu 2.0-505B scores within 1-2 points of Llama-3.1-405B and Qwen2.5-72B. The "6B active" gives 5× the inference speed of a 30B dense model with comparable quality.
The bigger takeaway: "Chinese open-source LLM" is at full-stack parity with Western open-source. The "Chinese LLM is 6-12 months behind" narrative is fully broken, and the "full-stack open-source" approach (model + training + inference + hardware) is becoming a Chinese strength. For the industry, this signals that the next round of LLM competition is in "full-stack integration," and Chinese vendors (Huawei, Alibaba, Baidu) are well-positioned.