Google disclosed that 75% of its new code was AI-generated in 2026; Microsoft put its own 2025 share at 20-30%. Whether that code is correct has benchmarks watching. How fast it runs, how much memory it peaks at, and how much energy it burns has mostly gone unmeasured at scale. A preprint posted to arXiv on 13 November by researchers at New York University Abu Dhabi and Universidade do Algarve, not yet peer-reviewed, puts that ledger on the table: PerfArena prompts 13 frontier LLMs to solve the same problems in 7 programming languages, then measures every one of the 9,455 correct solutions for latency, peak memory, and energy.

Three Bills Beyond Pass Rates

The premise is blunt. Existing code benchmarks, from HumanEval to LiveCodeBench, mostly count correctness; the few efficiency-oriented ones cover one or two metrics, few languages, and rarely energy. The authors had already quantified the problem in earlier work: LLM-generated Android apps carry roughly 10 times the density of performance anti-patterns of human-written equivalents, a self-reinforcing loop they call a performance loop of hell. Models learn from human code that rarely prioritizes performance, then mass-produce the same inefficiency.

An Arena That Borrows the Chatbot Arena Playbook

PerfArena organizes evaluation into two competitions. CLBGComp takes 10 compute-intensive classics from the Computer Language Benchmarks Game; LeetCodeComp curates 99 LeetCode problems whose accepted solutions already differ measurably in latency. Thirteen models - five from OpenAI, four from Anthropic, plus one each from Google, DeepSeek, xAI, and Zhipu - answer with the same prompt template in C++, C#, Java, JavaScript, PHP, Python, and TypeScript. Every solution runs on a Unix-like host under one protocol, with energy read straight from Intel RAPL hardware counters. The ranking mechanic copies Chatbot Arena: a cell that two models both solve is one duel per metric, the lower value wins, and ties within measurement noise split half a win each. The bank currently holds 9,455 correct, fully measured solutions. The paper also prices the exercise: one LeetCodeComp sweep averages $8.11 and 22 hours, with the priciest model at $19.56 and the cheapest at $0.56.

One Dominant Leader, Two Footnotes

The results are lopsided. Twelve models clear more than 90% of LeetCodeComp, so pass rates crowd together; performance is where they split. gpt-6-astra solves 99.7% of LeetCodeComp and 100% of CLBGComp, and no rival significantly beats it on latency, energy, or peak memory. The correct solutions of the other models average 1.1-1.6x its latency on LeetCodeComp and 1.2-3.6x on CLBGComp. Two footnotes matter. On LeetCodeComp, claude-opus-5-5 leads latency in Python and PHP while gpt-6-astra leads the other five languages. On CLBGComp, glm-5.3-flash and gemini-3.8-flash share Pareto level one with gpt-6-astra - but the paper itself notes glm-5.3-flash solves only 22 of 70 cells, too few for the tests to separate it.

Paying More Buys No Faster Code

The price story is the sharp part. Per token, gpt-6-astra at $5/$25 per million is not cheap (claude-fable-5-1 at $10/$50 costs most; gpt-5.6-luna at $0.2/$1.2 least). Converted to cost per correct solution, though, gpt-6-astra emits just 321 response tokens per solution on LeetCodeComp, costs less than half as much as the two most expensive models, and writes faster code. Per correct solution, the most expensive LLM burns 2.6x what gpt-6-astra does on LeetCodeComp and 9.3x on CLBGComp - and still generates no faster code. The conclusion reads plainly: beyond avoiding the cheapest tier on CLBGComp, where the four cheapest models happen to write the slowest code, paying more does not buy speed.

Energy Is Mostly Latency in Disguise

For anyone hoping energy would tell an independent story, the paper pours cold water: average power sits between 13.1 and 16.0 W for 90% of solutions, and energy rankings correlate with latency rankings at 0.91 and 0.97. Energy is roughly latency times a nearly constant power draw. The authors still report all three metrics separately, because task- and language-level disagreements do appear. Read the leaderboard with its limits in mind: everything ran on one quiet host with a 2-core Intel Core i7-7600U; CLBGComp has just 10 problems, and the paper concedes the statistical tests have little power there; the public leaderboard accepts results only from the official pipeline, though the code is open for anyone to self-deploy.

So the next time you pick a coding model, treat pass rates as the entry ticket and check the per-language leaderboard and the cost per correct solution. The leaderboard is live at perfarena.net and the code is open on GitHub. Flip the question around: if code speed has never entered your selection criteria, when exactly were you planning to read that invisible compute bill?