Pixel-space generative models learn directly from noise to pixels, sidestepping the separately trained autoencoder. The price is that a full-resolution image has to be represented as Transformer tokens. Most such models spend one token per spatial patch, which means a single grid decides three things at once: what the model reads, what each token predicts, and how many tokens join global attention. Want finer detail and the attention bill goes up with it. On a 256×256 image, dropping the patch size from 16×16 to 8×8 takes the token count from 256 to 1,024 and makes the quadratic term of dense attention 16× larger.

The problem is that coupling. What a token is allowed to communicate globally is decided by where it sits in the grid, not by what it finally produces.

Rewriting tokenization as budget allocation

A team from Peking University, Nanyang Technological University and SenseTime proposes SpecSlot, and it changes the split itself. The total image token budget is written as N = W × K: W is spatial anchor density, K is spectral subdivision inside each anchor. Every anchor expands into K slots, and each slot owns a fixed, disjoint band of DCT coefficients. A spatial residual is then added on the output side as a local correction. The generative state, the target and the loss all stay in pixel space.

Communication is assigned by role. In each anchor, the lowest-frequency slot is the carrier, and only that slot enters global attention; every other slot interacts locally inside its own anchor. The schedule is global, global, local, local, repeated three times, giving six global blocks and six local blocks of identical dimensions. The global sequence length therefore tracks W instead of W×K. In the ImageNet-256 configuration with s=16 and K=4, the global sequence holds W+C = 288 tokens, where a scheme that puts every slot into global attention needs 1,056. The corresponding quadratic term drops from 12(WK+C)²d to 6(W+C)²d + 6WK²d.

The paper summarizes this as tokenization as allocation. It loosens two things that were previously welded together: input density is set by the input grid, and output role is set by the frequency band inside an anchor. Neither has to hold the other hostage.

The advantage only opens up at 512 resolution

In the authors' own evaluation, SpecSlot W256/K4 scores 6.59 FID at 129.20 GFLOPs on ImageNet-256, only 0.09 better than the compute-matched JiT-CM control at 6.68. The paper says that gap sits inside the 95% interval across five random seeds, so it is essentially a tie. The real quality leader is still the more expensive JiT-B/8 at 4.34 FID and 218.50 GFLOPs.

At ImageNet-512 the gap opens. W256/K4 reaches 9.14 FID at 135.64 GFLOPs, 2.84 FID below the compute-matched control at 11.98 while using 3.9% fewer forward FLOPs. Against the costlier JiT-B/16 at 8.45 FID and 219.56 GFLOPs, the gap is only 0.69 FID, at 38.2% less compute and 12.69 against 7.93 img/s throughput. The authors' explanation is arithmetic: at 512 each anchor covers four times as many pixels as at 256, while the global sequence stays fixed at 288 tokens, so the extra detail stays inside the anchor.

For text-to-image at 256 resolution, DPG-Bench comes in at 62.21, 1.06 points above JiT-B/8 at 61.15, with forward compute dropping from 218.77 to 129.86 GFLOPs, a 40.6% reduction. The same model scores 26.99 on GenEval, below JiT-B/8 at 29.78. The architecture transfers to text conditioning without structural changes, and the price is giving up ground on a different alignment metric.

A detail worth not skipping: matching compute is not matching speed

The routing comparison table says the most about what this work actually is. In the same W256/K4 configuration, GGLL gives 6.59 FID at 129.20 GFLOPs and 9.06 img/s. The compute-matched dense-global control Dense-global-CM gives 6.81 FID at 128.96 GFLOPs but only 5.80 img/s. Measured forward GFLOPs are nearly identical, yet wall-clock throughput is far apart. Across sampling budgets of 20, 30 and 50 Heun intervals, GGLL throughput runs 1.47 to 1.49× the control.

One level further up, Full-global puts all 1,024 slots into global attention and posts the table's lowest FID at 5.84, at a cost of 225.82 GFLOPs, 74.8% above GGLL. So SpecSlot does not take the top score at equal compute. It is cutting 42.8% off the cost of routing everything globally while still holding a better quality than the compute-matched control. It picks the middle operating point, the one constrained by attention, not the leftmost one.

Mechanistic evidence: the low frequencies really do come back first

The paper supplies mechanistic evidence. Holding W256/K4/B1536 fixed and changing only which slot gets cross-anchor access: slot 0 gives 6.59, and swapping in slot 1, 2 or 3 gives 6.91, 7.21 and 6.97, all worse. Output-branch probes show the spectral term carries the larger self-energy, while the spatial term's share grows from 0.07 to 0.23 across the flow. Removing either branch at t=0.90 raises velocity MSE by 32.1× and 64.1× respectively, and at t=0.05 by 1.1× and 2.1×. The spectral scaffold is there from the start; the spatial correction is high-frequency top-up that arrives late.

Allocation ablations show that at a fixed slot budget of 2,048, more anchors means a lower score (7.28 for W64/K32, 6.28 for W256/K8, 5.66 for W1024/K2). With anchors fixed at 256, raising K from 2 to 4 to 8 gives shrinking returns, and the last doubling buys only 0.31 FID.

My read

All of these numbers currently come from a single origin: the paper, the official repository and the project page belong to the same team, every score is self-reported, and no third party has reproduced it. The paper itself is listed as under review at ICLR 2027. The right way to read it is as a methodological proposal, not as an established result.

The direction is still worth watching. Most of the past two years of pixel-space generation discussion has circled around how to remove the autoencoder entirely. This work swaps the question for how to decouple detail from communication budget. It does not beat the costlier JiT-B/8 at 256 resolution; what it wins is a non-dominated operating point in the constrained regime. The advantage clearly appears at 512, which hints that the payoff of decoupling grows together with resolution instead of spreading evenly.

For people building generative systems, the more immediately useful takeaway may be the side observation about controls: two models matched on compute can differ substantially in wall-clock speed. Cutting FLOPs is not the same as cutting latency, especially when what you actually change is the shape of attention rather than only its scale. What to watch next is whether this band division holds at larger scale, and whether the split between low and high frequency still makes sense on temporal data like video.

Paper and code: abstract page arXiv:2610.40647, project page yangjiongfan.site/specslot, repository JiongfanYang/SpecSlot.