As diffusion models moved from 2010s academic research to a hundred-billion-dollar industry in 2025, a hidden technical bet was ultimately decided by the evolution path of GPU architecture. The reason DiT (Diffusion Transformer) gradually replaced U-Net as the mainstream framework for image generation after 2023 isn't purely the algorithm's superiority, but because DiT's design happens to fit modern accelerators' compute characteristics — this is an industry story about "hardware lottery."
U-Net's design was finalized as early as 2015, and its core design philosophy targets the constraints of CPUs and early GPUs. The combination of skip connections and local receptive fields made U-Net excel on low-resolution image segmentation, but linearly extending this architecture to high-resolution image generation caused compute cost growth to far outpace quality improvement. Take Stable Diffusion XL as an example: its U-Net backbone already reached 2.6B parameters, but further scale-up didn't bring corresponding quality improvement, hinting at inherent scaling bottlenecks in U-Net.
DiT's core innovation is treating images as a sequence of 16×16 patches, processed with standard Transformer blocks. This initially seemed counter-intuitive — Transformer's compute complexity grows quadratically with sequence length — but at sufficient scale, DiT exhibited properties U-Net could not: generation quality continued to improve with parameter count, and training was more stable. The root of this difference lies in GPUs' deep optimization for matrix operations: Transformer architecture's matrix-multiplication ratio is far higher than U-Net's, allowing fuller use of modern accelerators' parallel compute capabilities.
"Hardware lottery" means a technology's success is largely determined by how well it matches the underlying compute architecture. U-Net's success in 2015 was no accident — it matched the hardware characteristics of its time. But as Transformers matured in 2017, diffusion-model designers faced a new opportunity: fully unleash Transformer's parallel compute for generative models. DiT's success confirms this path, while the failure of U-Net scale-up attempts based on diffusion confirms the same conclusion from the opposite side.
This case has implications for the entire AI field: algorithm innovation must synchronize with hardware development. Many seemingly leading architectures are eventually phased out not because of theoretical flaws, but because they're incompatible with mainstream compute platforms. Understanding this can help avoid "using a screwdriver to chop down a tree" mistakes in technology selection.