Running a multimodal model on an edge device means 4-bit quantization is not optional. But the real difficulty is not compute — it is the order in which the model breaks. A NeurIPS 2026 paper called ProQuant reports a counterintuitive observation: on the W4A8 quantized baseline of Qwen3-VL-2B-Instruct, the HellaSwag language benchmark still holds a 47.33% accuracy, while the AI2D multimodal-understanding benchmark has dropped to zero. The language prior withstood the quantization noise; the cross-modal alignment did not.
The paper is at https://arxiv.org/abs/2611.08216 .
What breaks is not language — it is "seeing"
ProQuant is the work of a team spanning The Hong Kong University of Science and Technology, Northwestern Polytechnical University, the Institute of Artificial Intelligence (TeleAI) at China Telecom, and Tsinghua University, authored by Yufei Xue and seven colleagues, and accepted to NeurIPS 2026.
The authors start with a diagnosis. When a multimodal model is quantized to low bit-widths, two capabilities degrade at once — but not to the same degree. The backbone retains a substantial linguistic prior, while multimodal understanding collapses entirely. The paper's reading is that cross-modal alignment is far more fragile to quantization noise than the model's intrinsic generative inertia.
That directly overturns the naive plan of restoring both capabilities together. The paper argues for decoupling them, and for putting multimodal understanding ahead of language modeling in the recovery order.
Two moves that run against intuition
One of them is self-relabeling (SRL). The original labels in open multimodal datasets do not match the generative distribution of the model being quantized: GPT-4-style captions are more literal and position-aware ("in the center of the table"), while Qwen-series models produce captions that are more holistic and interpretive ("creatively decorated donuts"). That stylistic mismatch introduces inconsistent supervision. So the authors discard the original labels entirely and let the target model regenerate its own responses.
The result is counterintuitive: self-labeling wins. In the ablation table, under W4A8, GPT-4 labeling scores 71.77, self-labeling 72.54, and a same-family 8B model 72.35. Under W4A4 the gap opens: GPT-4 manages only 56.78, self-labeling reaches 59.83. A bigger, stronger annotator from the same family still loses to self-labeling.
The other is progressive block activation (PBA). Training does not switch quantization on across all layers at once. It enables it first for the vision-sensitive layers while the rest stay in full precision, and only later brings the text-sensitive layers in as well. Which layers count as "vision-sensitive" is settled by a vision-sensitivity factor τ, computed per block as the average gradient magnitude over vision tokens divided by that over all tokens. The paper observes this factor peaking in the middle of Qwen3-VL-2B (28 layers): vision-sensitive blocks concentrate in the middle, text-sensitive ones sit toward both ends.
The ablations support that schedule. A logarithmic scheduler scores 72.54, ahead of cosine at 71.34 and linear at 71.00. Prioritizing by τ reaches 72.54, better than prioritizing by weight-gradient norm at 72.19, while random ordering manages only 71.88. On block granularity, treating one decoding layer as a single unit works best (72.54); coarsening to two layers per unit drops to 70.28.
The numbers in the tables
On Qwen3-VL-2B-Instruct at W4A8, ProQuant reaches an average accuracy of 72.54 — 6.60 points above the same-family QAT baseline LLM-QAT at 65.94 — while full-precision BF16 sits at 73.63, a gap of 1.09 points. The smoothing-based PTQ method SmoothQuant manages only 56.21.
The real dividing line is W4A4. There MBQ collapses to 6.37, LLM-QAT holds 27.29, and ProQuant reaches 59.83 against BF16's 73.63. On the same setting, MBQ's MME falls from 2029.18 to 0.00, while ProQuant still posts 1652.63.
Scaling up to Qwen3-VL-8B-Instruct, W4A8 gives ProQuant 79.80 against MBQ's 75.62, and W4A4 gives 66.93 against 5.76. Weight-only quantization was measured too: under W3g128, MBQ scores 53.21 and ProQuant 68.01.
At W8A8, however, ProQuant has no advantage — on Qwen3-VL-2B, MBQ scores 73.95 and ProQuant 72.88, with the PTQ baseline slightly higher. The paper's explanation is that strong PTQ is already sufficient at that bit-width, making the extra training cost of QAT hard to justify.
The hardware accounting
The paper evaluates on a TSMC 22nm-process ASIC, with on-chip latency and average power measured, off-chip DRAM simulated, and an end-to-end cycle-accurate accelerator model alongside. Across three backbones it measures up to a 3.64x prefill speedup and 5.64x energy reduction. Broken down for Qwen3-VL-2B: at W4A8, decode latency improves by 3.74–3.91x with energy down 3.76–3.93x, while prefill speeds up 1.58–1.76x with energy down 2.20–2.70x.
The authors also draw a line themselves: relative to W8A8, the roughly 2x and 4x peak compute throughput for W4A8 and W4A4 are hardware peak ratios, not end-to-end measured speedups.
One detail worth flagging
The abstract says code is available, linking to github.com/yufeixue-ai/ProQuant. As of this writing, that repository contains only .gitignore, a LICENSE, and a 56-byte README; its commit history is a single "Initial commit" from March 7, 2026. The implementation has not been released. The earlier abstract on the NeurIPS conference page still reads "code will be released upon acceptance." Anyone hoping to reproduce the results will have to wait.
So what
The value here is not the name ProQuant but the fact that the paper reorders the priorities of multimodal quantization. Collapse has a sequence, and so should repair — rescue the visual pathway first, and language capability follows on its own.
The finding that self-labeled data beats GPT-4 translates into every field that involves fine-tuning: aligning the supervision signal with the distribution of the model being optimized matters more than how strong the annotator is. Edge deployment will keep producing new quantization methods, but "which piece do you rescue first" may end up worth more than the specific operator.