Reasoning Arena turns RLVR waste into 41% faster training

As RLVR becomes widely adopted in large-model reasoning training, a long-overlooked problem has surfaced: when multiple trajectories sampled from the same prompt receive the same reward, the group-relative advantage is directly zero, and these samples are discarded for nothing. The Reasoning Arena framework proposed by Mistral AI and Cambridge's Han Zhou, Albert Q. Jiang et al. re-routes these "zero-gradient" samples to a judge system, mining fine-grained preference signals through trace tournaments (pairwise comparison), then uses a Bradley-Terry model to fit reward estimates from sparse comparison graphs, avoiding the O(n²) cost of full pairwise matching. Paper results show this mechanism improves an average of 7.6% over standard RLVR on competition math and programming benchmarks, training is 27%-41% faster, and generation compute is saved by nearly 50%. The base model is Mistral's own Ministral-3-8B-Instruct-2512, with DeepSeekMath-V2 and Qwen3 as judges. This upgrade of "outcome-oriented" RLVR to "process-aware" fine-grained signal recycling is a landmark sample of RLVR moving into the engineering efficiency era.