Running an open-weight model locally usually starts with one step: pull a pre-quantized file from a registry and use it. GGUF is the single-file format of llama.cpp, the inference engine Ollama is built on, and those registry tags are the first thing most people touch when they install a model.

But do those files actually still work?

A census that reached its third revision on October 29 (the paper) gives an uncomfortable answer.

A census covering 305 tags

The authors pulled every eligible code-capable GGUF tag from 15 model lines in the official Ollama library — 305 tags, 291 of them quantized — plus 22 popular community files on Hugging Face, for 327 artifacts executed in total.

Screening ran in two stages: a 15-task code screen flagged suspects, which were then rerun on all 164 HumanEval+ tasks.

Seven official tags turned out to be non-functional. That is 2.3% of the 305 official artifacts, spanning 3 of the 35 model variants. Six of them solved none of the 164 tasks; the remaining one, llama3.2-1B q3_K_S, solved 6. Every one of the seven was a low-bit quantization of a model with at most four billion parameters.

The authors also flag their own lower bound: running the full suite turns up eight more artifacts among the non-suspects that solve at most 16 of 164. So 2.3% is a floor, not the total.

The twist: the files are not corrupted

The obvious suspicion is a damaged download. The authors instead downloaded the original checkpoints, requantized them using the recipe recorded in each file's metadata, and compared tensor by tensor.

The four Qwen tags matched the official files across all 434 tensors; the phi3.5 tag matched across 197; and both sides produced byte-identical answers on all 15 tasks. Nothing was corrupted in transit.

The paper's phrasing: exact reproducibility did not imply function.

Three simpler explanations are ruled out one by one. Chat templates: the calibrated builds share the same template and work fine. Runtime: the failure reproduces on CUDA and Metal, on a separate machine, and on a later Ollama release. Tied embeddings: the working 1.5B model ties them too.

What is missing is calibration

Add an importance matrix (the imatrix in llama.cpp) and, holding weights, toolchain and tensor mix fixed, the numbers move: Qwen2.5-Coder-3B q2_K goes from 0 to 106–113 of 164 tasks, q3_K_M from 0 to 76–89, q3_K_L from 0 to 80–86. Third-party bartowski builds agree at 104, 81 and 77, and Qwen's own AWQ files score 102 and 125.

But q3_K_S stays weak under every calibration, at 13–30 of 164. It does not come back.

The size contrast is the more interesting part. The same uncalibrated recipe applied to the 1.5B model stays functional — 338 tensors matching, 13–15 of 15 tasks passing. The recipe alone does not predict failure; size does.

Screening without executing generated code

Perplexity is a cheap proxy: each of the five non-functional builds measured has 27 to 5,700 times the perplexity of its fp16 model, while the 4-bit tags stay within about 5% of it (4.2% and 5.1%). A threshold between 10 and 25 times fp16 flags all five without running any generated code.

The cost is bounded too: roughly 30 seconds of generation per artifact, and none of the 66 sampled tags that passed the screen was non-functional. The tool is out, named quantcheck.

The same wall, from two independent directions

This is not new. In a llama.cpp discussion from March 2025, someone reported that converting Qwen2.5-3B-Instruct to Q2_K produced nonsense on two different machines; that thread still has no answer.

An independent audit by Singh et al. (arXiv:2610.04403, submitted October 3) hit the same wall from another direction: 27 quantized artifacts across 13 families, from 0.35B to 14B parameters, pushed down to Q2_K, probed with 429 frequency-validated rare English words. At Q2 three fates emerge — total collapse into unusable builds, severe semantic dissociation in sub-2B models, and mostly robust preservation above about 3B. Parameter count dominates the damage (rho=0.72); tokenizer vocabulary size does not predict it at all (rho=0.12).

One tension is worth keeping. This paper proposes perplexity multiples as a cheap gate; the other audit finds WikiText-2 perplexity does not calibrate semantic damage — two artifacts with near-identical Q2 penalties (44.7% and 47.6%) split cleanly, one keeping its definitions (down 3.6%) and one losing them (down 43.6%). Perplexity can catch what has collapsed; it cannot catch what still speaks fluently but no longer knows what it is saying.

So what

The paper cites two measurements: an internet-wide scan found more than 320,000 publicly reachable self-hosted LLM services, including 155,423 Ollama instances; a year-long measurement saw 152,137 distinct IP addresses exposing Ollama. The ecosystem's default assumption has always been download-and-run.

This census shows a missing acceptance step. A 30-second generation screen plus an importance matrix can keep unusable tags out of the registry before publication; the paper's own recommendation is to screen low-bit tags before publishing, then calibrate or withhold the failures.

For your own setup: if a model pulled from Ollama emits gibberish, do not assume your GPU is at fault.