On May 12, 2026, a group of researchers from the Hong Kong University of Science and Technology (Guangzhou) and the Chinese Academy of Sciences released an impactful position paper posing a fundamental question: the way we evaluate LLM inference is fundamentally wrong.
Currently, academia and industry evaluate inference systems on accuracy, latency, throughput, and GPU utilization. But the paper points out that this only answers whether the model runs fast, not how many high-quality Tokens a system can actually produce under a fixed power and thermal budget. When inference is deployed at scale, the latter is the real production question.
The team draws on the Leontief production function from economics to construct the Token Production Function framework. The core insight: Token output rate is ultimately limited by the shorter of effective compute and delivered power, and system optimization is not a micro-engineering trick, but an energy lever acting on this production function.
The paper divides LLM inference history since 2020 into three stages: compute-abundant period, compute-explosion period, and the current power-wall period. April 2026 frontier-model API price quotes differ by as much as 10-30×; the paper argues this price spread is a result of different constraint choices — some paths choose to pile on compute, others choose to squeeze out maximum Token per Joule.
The most valuable insight is the recharacterization of techniques like KV cache compression, sparse attention, and quantization: these are not just engineering tricks that make the model more comfortable to run, but physically shift the boundary of how many Tokens per Joule of energy can be produced. Under a fixed power budget, Φsystem's combinatorial optimization can raise the throughput ceiling by an order of magnitude.
The paper's real contribution is a cognitive-framework shift: LLM inference is transitioning from a model problem to a heavy-industry problem. When context windows break a million tokens and data-center power becomes a scarce resource, whoever can produce more high-quality Tokens under a fixed power budget holds the initiative on the next generation of AI infrastructure.