Turn up the temperature to get more exploration, and long generations start to collapse into word salad. The counter-intuitive part is not that models fall apart, but that the paper's own summary puts it plainly: on the same model, at the same temperature, swapping one sampling rule can take a coding agent from solving 62 percent of its issues to solving none of them. A paper accepted to NeurIPS 2026 ran that experiment across the model zoo.
What the paper measured
The authors evaluated 29 instruction-tuned open-weight models between 1B and 122B parameters, spanning the Llama, Qwen, Mistral, Gemma, OLMo, Nemotron, GLM and gpt-oss families, with a core set of ten running from Llama-3.2-1B up to gpt-oss-120b on NVIDIA H200 GPUs. Every sampler lived inside a single vLLM logits processor that replaced native sampling, and the processor logged the candidate-set size at every decoding step. The benchmarks were ten established ones plus multi-key needle retrieval out to one million tokens; agentic coding used SWE-bench Verified with the mini-swe-agent harness and the official test harness; quality was scored by gpt-oss-120b on 1.5K-token windows. The paper is at https://arxiv.org/abs/2611.22148
The mechanism: temperature inflates the candidate pool
The explanation the paper offers is an ordering effect. Temperature flattens the probability distribution first, and only then does the truncation rule look for its cutoff on an already flattened distribution, so the cutoff keeps sliding backwards and the candidate pool keeps growing. Table 1 shows it in medians per decoding step: moving from temperature 1 to 2, top-p 0.9 goes from 2 tokens kept to roughly 90000, Top-H from 1 to 54, P-less from 1 to 21, min-p from 1 to 14, while top-nsigma stays at 1 because it cannot change with temperature by construction.
A sci-fi story from Llama-3.1-8B is more extreme still: top-p's pool grows from 4 tokens to 51845, most of the 128K vocabulary, just to reach 90 percent of the probability mass, while top-nsigma stays at 3. Across 133 combinations of sampler setting and temperature, the typical pool size predicts judged quality with a Spearman correlation of -0.77.
Why does longer context hurt more? The paper uses the entropy of the first answer token under greedy decoding: 0.02 nats at 8K, rising to 0.37 at one million tokens. The longer the context, the less sure the model is, and a rule that admits the tail turns that extra uncertainty into wrong digits.
How bad it gets
Coding agents show it most clearly. At temperature 2, agents running top-p, top-k or Top-H resolved 0 of 100 SWE-bench Verified issues, while the same agents running top-nsigma resolved 46 to 62, and they won every issue where the two samplers disagreed, across three Qwen3.5 models. On 100 fresh issues with no overlap with the first set, top-nsigma resolved 46, 62 and 53 percent on the 9B, 27B and 35B-A3B models while top-p and Top-H resolved none, for records of 46-0, 62-0 and 53-0.
The configuration closer to practice is harsher: with a 500-step limit, 262K context and 50 issues, at temperature 1.5 top-p and pure sampling both score 0 percent, min-p 48 percent, P-less 58 percent and top-nsigma 52 percent, with win/loss records against top-p of 24-0, 29-0 and 26-0, all clearing a two-sided sign test. At temperature 1.0, no difference between samplers is significant. Scaling up the model does not help: Qwen3.5-27B and Qwen3.5-35B-A3B at temperature 2 resolve 72 percent and 58 percent under top-nsigma, and nothing at all under top-p, top-k, Top-H or pure sampling.
Reading alone breaks the same way. With 16 key-value pairs hidden in PG-19 novels and contexts out to one million tokens, at temperature 1.5 top-nsigma retrieves 75 percent, P-less 78 percent and min-p 73 percent, against 59 percent for top-p and 48 percent for pure sampling. On LongBench v2 at temperature 1 top-nsigma scores 39.5 percent against top-p's 36.6 percent, and at 1.5 top-p drops to 25.6 percent. Pooled across benchmarks, the reported median is that top-nsigma retains 100 percent of its temperature-1 score while top-p retains 26 percent.
The order of temperature and truncation
An implementation detail is easy to miss here: the paper notes that HuggingFace, vLLM and SGLang all apply temperature before truncation by default, while llama.cpp defaults to the opposite order. I went to the llama.cpp source to check, and the default sampler chain in common_params_sampling runs penalties, dry, top_n_sigma, top_k, typical_p, top_p, min_p, xtc, with temperature last (https://raw.githubusercontent.com/ggerganov/llama.cpp/master/common/common.h). On that chain truncation happens before temperature, the reverse of the other three.
How much does that ordering matter above temperature 2? The paper runs a temperature-last ablation: if the candidate set is picked at temperature 1 and temperature is then applied only inside that set, Top-H, min-p and P-less hold their temperature-1 quality all the way to temperature 3 (4.1 to 4.6), where the default order leaves them at 1.1 to 1.2. On LongProc, min-p 0.1 scores 25 percent at temperature 2 instead of 15 percent, and top-p 0.95 comes back from 1 percent to 25 percent at 1.5.
The cold water
The paper's own limitations are stated plainly. Every quality score comes from a single LLM judge (gpt-oss-120b), no human evaluation was run, and LLM judges are known to be biased by length, style and position. Coverage is ten open-weight models from 1B to 120B, the agentic study uses one harness, and most grids run on a 9B model. Closed models cannot be tested at all, because providers do not let users control the sampler. Each forced-length cell has 80 generations, which is not enough to resolve differences of a few tenths of a judge point, and the authors do not claim to resolve them.
Conflicts of interest are worth flagging too: min-p is the authors' own prior work (Nguyen, Roush and Shwartz-Ziv all appear on that ICLR 2025 paper's author list), and they previously wrote DRY and XTC. The sampler that actually wins here, top-nsigma, comes from a different team. That makes the conclusion more credible than self-promotion, but every number still has a single origin, and the work was funded by Thoughtworks.
So what
The mechanism has independent corroboration. Tang and colleagues, who proposed top-nsigma, pointed out back in ACL 2025 that probability-based metrics like top-p and min-p are inherently sensitive to temperature scaling, while removing noise directly in logit space gives temperature-invariant token selection (https://aclanthology.org/2025.acl-long.528/). P-Less at ICLR 2026 reports that its hyperparameter-free sampler degrades much less as temperature rises (https://mlanthology.org/iclr/2026/tan2026iclr-pless/). La Rosa's September 2026 paper tested thirteen open-weight models on GSM8K and MMLU-Pro at temperatures 0.7, 1.0 and 1.3, and concluded that truncation samplers pay off mainly once higher temperatures have already degraded the model, with none of the tested truncation samplers beating plain temperature sampling where accuracy stays stable across temperatures (https://arxiv.org/abs/2609.15476). Three independent lines of work point the same way.
The practical advice is unglamorous: if you turn up the temperature, use a sampler that leaves the tail out, and top-nsigma with n of 1 or less is already available in llama.cpp. Report the realized candidate-set size per step instead of nominal hyperparameters. When we evaluate models we watch parameter count, context length and benchmark scores, and almost nobody writes down the temperature and truncation order in the report. Which published number do you think gets rewritten next?