Asking a language model to write down a handful of 'random' numbers is not, in fact, random. That observation has been validated repeatedly over the past two years: the distribution of numbers models produce deviates from the distribution they were instructed to sample, and different models show relatively stable preferences over value ranges and last digits. That observation has opened a technical path: do not interrogate the model with questions, just make it count, then treat the resulting string of digits as a fingerprint to check which model actually sits behind an API.

A recent paper from a team at the Faculty of Computing, Harbin Institute of Technology pushes that path somewhere new. The paper introduces a benchmark called NumPrint. Using prompts in Chinese, Japanese, and English, it asks models to write out 294 integers between 1 and 355 (inclusive) in a single response. The full dataset covers 51 models and 5,036 answers. The paper is at https://arxiv.org/abs/2611.14010

From statistical preference to sequence dependency

The paper's argument is that the information carried by a digit sequence does not stop at the level of 'which numbers show up most often'.

The authors split the representation into two branches. The first is a statistical-feature encoder that compresses a sequence into 944 features: distributions over integer values, over digits, and over remainders, plus positional blocks, increments, transitions, lags, and spectral summaries. Those feed a single-hidden-layer MLP with 1,024 hidden units that emits class scores directly. The second is a causal attention encoder: each integer becomes a 64-dimensional embedding, passes through a single-layer, single-head causal self-attention so that position t can only attend to t and everything before it, and a learned weighted readout then pools the whole sequence into a 64-dimensional vector.

Both branches feed an adaptive classification module. It applies temperature calibration to each encoder, combines them through a gate weighted by confidence, and finishes with a per-class linear correction. The two encoders are trained separately, and five-fold cross-fitting produces out-of-fold predictions used to fit that fusion module, so the classifier is never scored on data its encoders had already seen.

The ablations show how the design's weight is distributed. Removing the statistical encoder drops accuracy from 92.72% to 87.43%, a loss of 5.29 points. Removing the causal encoder costs only 1.45 points. What actually carries the identification is 'which numbers this model likes to write'; 'how those numbers connect to each other' is a supplement.

The numbers

On 756 test answers in the 51-class closed-set setting, a single sequence reaches 92.72% accuracy and 91.10% macro-F1. In the same table MLP and LDA both reach 91.27%, which SMCT leads by 1.45 points. The authors' own reimplementation of the open-source lm-detector project, using its 429-dimensional representation, reaches 88.89%, which SMCT beats by 3.83 points. Random forest manages only 68.39% on this task.

The effect of sequence length is monotonic: given only the first 32 integers, accuracy is 50.26%; at 64 it is 70.63%; at 128, 83.60%; at 256, 91.40%; only at the full 294 does it reach 92.72%, after which the curve visibly flattens.

The family-level results are more interesting still. The 51 models group into 17 families by developer. Family-level identification accuracy is already 97.62% from a single sequence, then 99.54% from two, 99.98% from three, and 100.00% from either four or five. Of the 55 model-level errors at a single sequence, 18 cross family boundaries, or 32.73%; by four sequences cross-family errors are zero, and all 169 remaining errors fall inside a family. That is where the hard part lives: gemini-3.7-flash is misidentified as gemini-3.8-flash 57.1% of the time at one sequence, and glm-5.2 is read as glm-5.3 58.3% of the time. Sibling models from the same lab share training recipes and sampling habits anyway, and that is precisely what makes them difficult.

The ceiling, as the authors state it themselves

The paper's Limitations section is honest, and all three points are worth carrying over.

First, NumPrint covers 51 known models; identification happens inside that closed candidate set. Handling models outside the set would require additional unknown-model detection and updates to the reference set.

Second, collection conditions were not independently controlled. System prompts, generation settings, reasoning modes, and conversation context can all change numerical fingerprints, and the paper draws on mixed data collected across multiple languages and multiple interfaces.

Third, the sequence-length experiment takes prefixes of already-collected answers and retrains on them, rather than requesting sequences of different lengths.

There is a fourth point the paper mentions but deserves amplifying: this approach depends on the model's inability to sample uniformly in the first place. If a model someday does sample random numbers properly, the fingerprint goes with it.

More than one team is on this

Two other independent lines of work point in the same direction at the same time.

Bruckner's One Token Is Enough (arXiv:2607.10252) drives the cost down further. With a one-sentence prompt asking for a random number between 1 and 100, collected across four languages at one output token per query, it measures 165 models served through OpenRouter. It finds these distributions highly non-uniform (median cell entropy of 1.0 bit), and that Jensen-Shannon divergence between fingerprints recovers model lineage with 59.5% leave-one-out accuracy against an 18.4% chance rate; a biometric-style protocol reaches a 7.3% equal error rate over a full 40-cell battery. That paper also reports an ecosystem anomaly: an endpoint carrying a proprietary flagship brand whose distribution is indistinguishable from an open-weight Qwen model.

FLIPS (Richardeau et al., ICML 2026, arXiv:2606.03330) works with binary pseudo-random sequences and aims to distinguish instance-level configurations of the same model. Change the temperature, the quantization, or the system prompt and you get a different fingerprint. The official code repository reports 96% closed-set and 90% open-set accuracy across 237 model instances, against 35% for an adapted LLMmap baseline. The NumPrint paper also notes that if FLIPS is compressed to a single query, closed-set accuracy drops below 70%. That single-query regime is exactly the hard one.

Three routes, one common property: none needs logprobs, none needs white-box weights, none needs cooperation from the model vendor. All you need is the ability to send a request and read the output.

So what

If the cost of obtaining a fingerprint is a single API call, the confidentiality barrier around 'which model is this' has largely been torn down. For aggregators and API resellers, swapping a claimed model for a cheaper sibling in order to pocket the difference now has a nearly zero-cost detection path. And for model owners, the fingerprint is no longer under their sole control.

More subtle is the reverse tension: this string of digits is identifiable precisely because the model cannot sample truly randomly. If a vendor fixes the uniformity of its sampling, the identity goes with it, and that is exactly the direction in which vendors selling more obedient and more controllable models are already pushing.

So the real question is not whether the fingerprint is accurate. It is whether a model's behavioral deviation should be treated as a bug to be fixed, or as an asset worth keeping verifiable.