For large-model inference, multi-sampling + voting strategies are already common — Best-of-N, Majority Vote, Pass@N — all familiar to anyone following LLM progress. But one long-overlooked issue is a fundamental misalignment between training objectives and inference strategies. Standard fine-tuning (SFT) and reinforcement learning (RL) both optimize for single-sample accuracy, never considering that the model will ultimately be used in an "ensemble" fashion. A recent paper from the University of Michigan proposes Compute Aligned Training (CAT), attempting to address this from the training stage.
Where's the problem?
Take Pass@100 as an example. Suppose the model already has a 50% chance of being correct; continuing to push that probability from 50% to 90% yields almost no additional Pass@100 gain — success was already locked in. But standard training continues pushing the gradient in that direction, causing "effective over-training." Conversely, Majority Vote is a competitive process: an answer with p=0.35 wins as long as it's higher than the strongest competitor's p=0.1, but standard training would push the model toward p=1.0, pursuing "overwhelming victory." These are two completely different optimization logics, but today's model training ignores both.
How does CAT do it?
The core idea is elegant: formalize the inference-time strategy (e.g., Best-of-N, Majority Vote) as an operator acting on the base policy, and use the post-operator distribution as the training target. Concretely, CAT derives new gradient reweighting mechanisms for SFT and RL, assigning gradient weights according to each sample's "marginal contribution" to the final aggregated result, rather than naïvely doing gradient descent on the answer probability itself.
The paper validates the effect in three scenarios: in LLM SFT and RL, CAT significantly improves Pass@N and Majority Vote performance; beyond LLMs, CAT also works on protein language models (PLMs); on policy generalization, the CAT framework can be compatible with any inference strategy, not limited to a specific one.
What does this mean?
Test-time scaling is already an important direction for boosting model capability, but previous efforts have focused mainly on the inference side — longer CoT, more samples, more complex verification chains. CAT's value lies in pointing out: if the training stage can let the model "understand" that it will be used in ensemble form, the inference-side investment will be amplified, and efficiency will be higher.
This isn't a complex engineering trick, but a conceptual alignment — shifting the model from "aim to be right every time" to "aim to be right after ensembling." As more open-source models support local inference and batch sampling, the impact of this training paradigm may arrive earlier than many expect.