On June 8, 2026, Xiaomi MiMo and TileRT jointly released MiMo-V2.5-Pro-UltraSpeed, pushing the generation speed of a trillion-parameter (1T) model past 1000 tokens/s for the first time on a standard 8-card general-purpose GPU node, peaking at about 1200 tokens/s. This is not another leaderboard press release, but a clear challenge to the path of large-model inference.
The real value of 1000 tokens/s is turning "fast" into "deep." In the same wait window, the model can try dozens of reasoning paths in parallel (Best-of-N, Tree Search), the Coding Agent loop latency is compressed to sub-second, and high-frequency trading, anti-fraud, real-time dialog, and other scenarios can finally accommodate 1T flagship models into millisecond-level decision loops.
The implementation path is not bigger clusters, but extreme model-system co-design: on the model side, only the MoE experts are subjected to FP4 (MXFP4) quantization + QAT, with the rest of the modules retaining precision; DFlash block-level masked parallel prediction is introduced as draft, naturally aligned with MiMo-V2's built-in sliding window attention, and the accepted length in Coding scenarios reaches 6.30; the Muon second-order optimizer + self-distillation push draft training to the limit. On the system side, TileRT uses a persistent engine kernel + warp specialization to push operator boundaries down to the microsecond level, with compute kernels tailored to the quantization and speculative-decoding pipeline.
Compared to Cerebras' wafer-scale integration and Groq's on-chip SRAM custom chip, this general-purpose GPU + co-design path avoids the prohibitive hardware barrier. The FP4 QAT'd MiMo-V2.5-Pro-FP4-DFlash is already open-sourced on Hugging Face, and the community can directly reproduce it. When 1T models run at 1000 tokens/s on general-purpose GPUs, the statement that "frontier models can only run in big-tech data centers" needs a question mark starting today.