OpenAI's self-designed inference silicon has been rumored since last year, but on August 25 at Hot Chips 2026, Richard Ho's team finally put numbers on the board. This is OpenAI's first real attempt to answer two questions: how much faster is a custom ASIC than Nvidia Blackwell, and whether a nine-month tape-out miracle is held together by AI writing its own kernels.

The Real Numbers in Three Charts

OpenAI did not pick its strongest internal model. Instead, it ran three public models through SemiAnalysis's InferenceX benchmark so anyone can reproduce the results:

  • GPT-OSS 120B: peak throughput 85,448 vs 44,960 mixed tokens/s per kW (vs GB200), a 1.9x lead; end-to-end latency dropped from 1.80 s to 1.03 s.
  • DeepSeek R1 670B (MXFP4): 19,641 vs 11,781 mixed/kW, 1.7x; latency 1.65 s vs 5.99 s.
  • Kimi K2.5 1T (MXFP4): 18,195 vs 11,862 mixed/kW, 1.5x; latency 1.56 s vs 5.31 s.

The per-user story is even sharper. On DeepSeek R1, Jalapeño pushes 700 tokens/s per user vs 169 on the comparison system. On Kimi K2.5: 694 vs 182. On GPT-OSS: 1,459 vs 535. In other words, the "is it snappy" gap reaches over 4x at the user level.

Source: OpenAI's official disclosure page and TechCrunch's report.

The 700-Watt Tradeoff

The ASIC carries a 700 W TDP rating, with measured sustained power "at or below 550 W," while the comparison systems (GB200 / GB300) sit at 1,200 W / 1,400 W. Power is halved, single-chip TDP is roughly half of Blackwell, but per-user throughput is 2–4x higher — meaning OpenAI is trading system-level co-design for performance: less data movement, larger network domains, and explicit placement of the KV cache.

Richard Ho drew a clear line himself: all numbers are single-turn, 8k context, InferenceX. No AgentX, no multi-turn, no prefix-cache-heavy workloads. Jalapeño today proves "cheap LLM inference" can dominate. The agent workloads that actually pay the bills are still unverified. That is also why SemiAnalysis's own postmortem at Hot Chips 2026 called the Jalapeño vs Blackwell comparison "unfair and incomplete" — Nvidia's next-generation Rubin is already shipping with the same HBM4 memory and 900–1,150 W power envelope, which is the real opponent Jalapeño will have to face.

The More Important Story: AI Writing Kernels

The hardware is surface-level. The disclosure that made practitioners sit up:

"Using Codex with GPT-Astra, the team brought three open-weight models that were not part of Jalapeño's original production plan to high performance within two months. For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the existing human-expert-written implementations."

Translate this: from silicon return to production-ready models, Jalapeño took less than nine months; part of the GPT-OSS attention and MoE block kernels were auto-written by Codex plus GPT-Astra, and on selected blocks ran 1.5–1.8x faster than human-expert kernels. OpenAI is not offering PR copy, it is offering a quantifiable evidence chain — AI can not only write code, it can write code that beats veteran GPU kernel engineers.

If this curve scales, the decade of accumulated CUDA moat has to be repriced. OpenAI's single-chip win is bounded; what matters is the bidirectional coupling between "models" and "hardware." Once it exists, any new model architecture can ship at silicon speed within nine months, an iteration cadence Nvidia cannot match. The CUDA moat was always "humans writing kernels at scale," not "the code itself." Once the writing is done by models, the accumulated ecosystem amortization disappears fast.

Does Nvidia Panic?

OpenAI itself hedged: "we will continue to deploy NVIDIA and other partners' accelerators at scale." Jalapeño is not designed to replace Nvidia; it gives OpenAI's own API another lever to push down cost-per-token. By their estimate, if inference cost drops 50%, high-frequency agent workflows no longer hit the wall of "too expensive to use."

SemiAnalysis itself noted earlier that the Jalapeño vs Blackwell comparison was "unfair and incomplete." Against shipping Nvidia Vera Rubin (same HBM4 memory, 900–1,150 W), Jalapeño lands at rough parity on TCO, not a wipeout. The CUDA ecosystem still has value; the "only choice" window is just closing. After TPU, Trainium, and Jalapeño, frontier labs are no longer locked into "Nvidia only," and pricing power starts to shift.

What This Means for Developers

If you call the OpenAI API today, you change no code. Jalapeño is internal infrastructure, not a card you can plug into a server. The real payoff arrives in 2027, when the next API pricing or rate-limit adjustment reflects the cost structure. Until then, three signals are worth tracking:

  1. The evolution curve of Gen 2 and Gen 3 (OpenAI has publicly confirmed both are in development);
  2. Whether Jalapeño holds the Pareto frontier on AgentX long-context, multi-turn, and prefix-cache-heavy workloads;
  3. Whether "Codex writes kernels 1.5x faster than humans" generalizes from selected blocks to the full chip code base.

If all three happen at once, 2027's LLM inference market will not be a binary "Nvidia vs custom silicon" fight — it will be a generational contest between vertically integrated model-hardware stacks and general chips plus legacy compiler moats.

Reference sources: