RLVR - reinforcement learning with verifiable rewards - quietly assumes the verifier can tell right from wrong. Math and code are easy precisely because answers can be machine-checked. Outside those comfort zones, a complete verifier is often impossible to build: all you have is public tests, format constraints, or another model looking for flaws. A NeurIPS 2026 Spotlight paper from Nanyang Technological University and Skywork AI (arXiv:2611.17287) formalizes this predicament as one-sided verifiability (OSV), and its fix is counterintuitive: train only on negatives, reward no 'pass' at all.
Rejections are trustworthy, acceptances are not
The paper first measures the asymmetry. Sampling with Qwen3-4B on TACO, 64.5% of solutions that pass the public tests turn out wrong against the full test suite (66.8% for Qwen3-8B). The reason is plain: training sees an average of 1.7 public tests per problem, while evaluation uses a full suite averaging 204. On CodeContests the false-positive share among accepted solutions is 20.7% (1.9 tests vs 12); ZebraLogic constraint checks sit at 73.0%, and an LLM self-verifier at 46.3%. In the other direction, a 'rejected' verdict is almost always genuinely wrong: negative precision is 100% by construction for tests and constraints, and 95.8% for the LLM self-verifier.
The paper's phrasing is blunt: once false positives are used as rewards, they become attractors of the policy optimization dynamics - shortcut solutions that slip past partial checks collect positive gradients, so the model learns to raise its pass rate instead of its correctness.
NOPO: treat 'passed' as unlabeled
The method, Negative-Only Policy Optimization, uses the verifier only for the evidence it can safely give: high-confidence negatives. Per sample, an NPO-style suppression loss pushes down negative responses relative to a reference policy, with a gradient that self-attenuates once suppression is done. Per group, a support gate skips any prompt whose 16 rollouts contain zero non-falsified responses, avoiding directionless 'move-away' updates. Passed samples are never treated as correct demonstrations; they only count. Training is light: verl plus vLLM, eight A100 GPUs, about 12 hours, 150 steps.
First on average, and the truth shows at large k
Per the paper's own measurements, average pass@16 across five benchmarks rises from 46.85 (base) to 52.86 on Qwen3-4B, with GRPO at 48.93; on Qwen3-8B it is 52.13 versus 48.27. NOPO takes nine of ten benchmark-backbone cells. The harder evidence is at large k and in transfer: on CodeContests at k=512, NOPO reaches 66.95 and the base 51.05, while GRPO manages only 48.54. The paper notes GRPO has the best pass@1 of all methods (37.23) yet falls below the base model at k=512 on both benchmarks - sharpening, not frontier expansion. In transfer, a model trained on CodeContests averages 29.49 on unseen benchmarks (base 23.66, GRPO 23.30 - below base). At the 512-sample budget NOPO keeps 120 of the 122 problems the base solves and adds 40 new ones.
There is also direct reward-hacking evidence: after GRPO training, the false-positive share among accepted solutions rises from 20.7% to 25.2% while negative recall drops from 91.3% to 79.8%; NOPO pushes the false-positive share down to 16.6%.
The mechanism is not a one-off
'Negatives carry the useful signal' is not this paper's private discovery. The baseline it compares against, w-Reinforce (arXiv:2506.01347, Zhu et al., NeurIPS 2025), independently reports that training with only negative samples - on Qwen2.5-Math-7B, Qwen3-4B and Llama-3.1-8B-Instruct, with k up to 256 - often matches or surpasses PPO and GRPO, while reinforcing only correct responses lifts pass@1 but hurts large k as diversity collapses. The two teams differ in treatment: w-Reinforce still gives passed samples a small weight of 0.1; NOPO treats them all as unlabeled.
The paper's own limits deserve mention: negatives say where not to go, not where to go. On OlympiadBench with Qwen3-8B - the one cell NOPO loses - it trails GRPO by 0.45 points, which the authors attribute to negative recall of only 8.8% there: too few trusted negatives.
So what
If you are building your own verifier for RL, two takeaways. First, do not equate 'no error found' with 'correct' - that is a bonus program for shortcut solutions. Second, do not judge by pass@1 alone; large-k and cross-benchmark transfer are what separate genuinely stronger models from merely better test-passers. Cheap checks make reliable rejections; uncertain acceptances can stay on the shelf. A one-sided verifier is not a broken verifier - it is an underused signal source.