There is a habit in image generation of talking about parameter counts and leaderboard scores. On November 4, a two-person lab called Linum published a different kind of number: the bill. Their text-to-image model, Pyramid-JiT, was trained for a total of 8,415 H100-hours, which the lab prices at $21,038. The weights and the code are released under Apache 2.0, and the full training recipe — sample counts, batch size, optimizer split, noise schedule, learning-rate curve — is published alongside them (field notes: https://www.linum.ai/field-notes/pjit-21k).

The architecture is the more interesting part of the story, and the reason a two-person team would bother publishing a bill at all.

Throwing away the VAE

Most text-to-image systems work in latent space: a VAE compresses an image into a lower-dimensional latent, and the diffusion model learns to generate inside that subspace. Linum argues this route hits an empirical ceiling. Their claim is that 16x16 token compression is about as far as latent diffusion models go without surgery on the autoencoder, and that improving a VAE’s reconstruction quality can make the downstream generations worse rather than better.

So they removed the VAE. The diffusion model operates directly on pixels and learns a latent space purpose-built for generation. The price of that choice is that each token carries more information, so patches grew from 16x16 to 32x32: a 32x32x3 = 3,072-value RGB patch is projected into 1,024 dimensions, a 3:1 compression rate, and the attention context window shrinks by a factor of four relative to a 16x16 setup. Their previous model, Linum v2, was a fairly standard Wan VAE plus DiT with 16x16 token reduction, the same as most models.

The architecture, Pyramid-JiT, runs 22 DiT blocks in the trunk, preceded by two image refiner blocks and two text refiner blocks, with a hidden size of 2,944 and 23 heads of width 128. Text comes from a frozen Qwen3.5-4B encoder, read at layers 7, 15 and 27. Sampling uses 50 Euler steps with adaptive projected guidance at scale 15, and the released checkpoint is 8.7 GB in fp32 alongside an 8 GB bf16 text encoder.

Letting the trunk draft its own answer

The more unusual design choice is the readouts. Because the model is trained with x-prediction in pixel space, every intermediate DiT block is already implicitly predicting a version of the final image. Linum exploits that: a two-layer PixelREPA readout at block 5 aligns the model’s hidden state to DINOv3-L features, and two linear readout heads after blocks 10 and 16 predict the clean image at quarter and half resolution. These heads are training-only and add about 0.24B parameters, bringing the training-time count to roughly 2.42B against 2.18B at sampling time.

In their October 6 architecture post, the comparison is direct. On the same 22-block trunk with the same recipe for 75M samples, a single readout at 512x512 gives a validation loss of 0.1037. Switching to three readouts — 128x128 at block 10, 256x256 at block 16, 512x512 at block 22 — drops it to 0.0934, a 10.0 percent reduction. Three readouts at full resolution give 0.0948, an 8.5 percent reduction, and down-weighting the low-resolution readouts gives 0.0983, a 5.1 percent reduction. The same post claims P-JiT matches Linum v2’s final FD-DINOv2 score using 11.3 times fewer training samples. That is a comparison against the lab’s own previous model, not against other labs’ models.

What the bill actually contains

Pretraining cost $20,348, which is 8,139 H100-hours priced at $2.50 per hour. Post-training cost $690, or 276 H100-hours, using a variant of Finite Difference Flow Optimization with JEV-Image and human-in-the-loop judges. The hardware was 64 H100s across 8 nodes, in bf16 with FSDP2, 32 images per GPU. Training ran in two stages: 610.27M samples (297,985 steps) at 512-class resolutions, then 49.97M samples (24,400 steps) at 1024-class resolutions, with a global batch of 2,048 throughout.

The optimizer is split. Muon, with momentum 0.95 and Nesterov, handles the 2D block matrices; AdamW handles everything else; and the Muon updates are RMS-matched to AdamW. The learning rate peaks at 2e-4 with a linear warmup over 1M samples, stays flat to 550M, then decays as 1 minus square root down to 2.5e-5 over the final 60M. The noise schedule runs a curriculum: LogitNormal(0.8, 0.8) for the first 80M samples concentrates training at high noise so structure comes first, then LogitNormal(-0.2, 1.0) widens the distribution so fine detail can be learned. When the model moves to 1024, the noise scale doubles from 2 to 4, and RoPE interpolation aligns the 1024 patch indices back to the 512 coordinate axes.

The weaknesses, stated plainly

The model cannot write text or generate posters. That is not an oversight: images of rendered text — infographics, PDFs, book pages, slides — were filtered out of the dataset on purpose, because the lab does not consider that data useful for animation. The prompt upsampler returns a text_requested error for such prompts instead of a caption.

Other limits are listed explicitly: full-body shots, many entities such as six cars or crowds, interactions between several people, and world knowledge. The released failure examples show legs stretched into each other, an extra pair of hands under a person’s legs, and a die where every face shows six dots.

On prompt-following benchmarks, the model is not the strongest in its own family. On raw prompts, GenEval scores are 0.561 for Linum v2, 0.641 for JiT-DDT and 0.612 for P-JiT. Once prompts are rewritten into the training caption style, the ordering flips: 0.855, 0.808 and 0.817 respectively. DPG-Bench lands at 84.05 for P-JiT on raw prompts and 84.31 on upsampled ones, against 84.79 and 84.54 for Linum v2. The lab’s own explanation is that pixel-space models produce better aesthetic samples and match the underlying image distribution more closely, but appear relatively undertrained on aspects of prompt following such as counting — on the counting task, upsampled GenEval gives 0.80 for Linum v2 against 0.56 for P-JiT.

Why publishing the bill matters

First, the recipe is copyable. It is not just weights and inference code: how the noise schedule is staged, how Muon and AdamW are split, the fact that EMA pauses for 250 steps when the gradient norm exceeds 10 times its rolling median — a spike-aware EMA — these are details that are usually kept private.

Second, this checkpoint is explicitly a stepping stone. The stated goal is cheaper video generation, because video models almost always start as image models: they have to learn nouns before verbs. Linum notes that a short 720p five-second clip in its v2 model costs 110K tokens of attention, while LLMs train on context sizes of 8K or less for 97 percent of their training regime. Attention is O(n^2) in context length, so cutting token count is where the savings come from. That is why this run was done at all — the lab says the results surprised them for a model this size, and that they are releasing it to encourage others to explore efficient training methods.

The Hacker News thread drew 95 points and 9 comments, where readers pointed at over-smoothing, an uncanny sheen and mangled limbs, and asked how much cherry-picking went into the showcase. The repository carries 8 stars and a single commit, and both the model card and the README were written by Claude Opus 5.5 on Linum’s behalf, with the model, the training and the code review remaining Linum’s own work.

So what? The $21,038 is not the headline. What matters is that the bill and the recipe arrived together, which turns "can I train a usable text-to-image model myself" from folklore into an engineering question with a documented answer. The more interesting question comes next: if this pixel-space recipe actually transfers to video, is the attention savings enough to drop the price of animation tooling by an order of magnitude?