arXiv 2606.26671 introduces NebulaExp-8B, an 8B-parameter model from ZTE focused on "fully reproducible post-training." The paper's biggest contribution is not the model itself but a complete engineering blueprint: an OPD (On-Policy Distillation) pipeline that combines multi-teacher distillation with on-policy self-improvement, packaged into a recipe any team can replicate.
The training pipeline has three stages: SFT on public instruction data → multi-teacher on-policy distillation (using Qwen2.5-72B and Llama-3.1-70B as dual teachers) → RLHF with on-policy sampling. Each stage has a full open-source configuration, including learning rate, batch size, data ratio, and reward-model design.
The result: NebulaExp-8B scores 67.3 on MMLU and 78.5 on HumanEval, on par with or surpassing the open-source 8B SOTA. The most important thing is "full transparency" — every training script, every data preprocessing step, every hyperparameter is open-sourced. Anyone with 8 GPUs can reproduce the result.
The bigger signal: NebulaExp-8B represents the new trend in Chinese LLM open-source — moving from "open the weights" to "open the full pipeline." For the industry, this is a significant shift: the "we can only replicate closed-source models" pattern is being broken, and more "fully reproducible" base models will appear.
For ZTE, the strategic value is also clear: 8B is the sweet spot for edge deployment, and a fully reproducible 8B recipe means ZTE can offer a turnkey LLM solution for telecom customers, with the training methodology as a moat.