When a company that has been building mainframes for more than sixty years is willing to put a non-native ISA like Arm into the core — and not as a sidecar — it deserves a closer look. At Hot Chips 2026, IBM showed the next-generation IBM Z and LinuxONE processor, and the design choices are anything but conventional.

One core, two ISAs — no stitching

By "dual-ISA," IBM means every core can natively execute both z/Architecture (the mainframe's long-standing instruction set) and AArch64 (Arm v9.3). The keyword is "native": IBM did not take the translation or emulation shortcut. Instead, AArch64 is implemented directly in hardware, with a little-endian Arm sitting alongside a big-endian z/Architecture, plus SVE/SVE2 support. In total, 2,792 AArch64 instructions are implemented — more, IBM quipped on stage, than the number of z/Architecture instructions, so stop calling RISC "reduced."

Two implementation details stand out. First, the design ingests Arm's published XML architecture descriptions to feed the decoder automatically, while the dispatch and issue logic reuses the Z core's register-rename machinery, just remapped onto the GR16-31 range. Second, AArch64 is treated as a first-class citizen, not a compatibility afterthought. The result, IBM says, is that Arm software runs unmodified and out of the box, and that the core is Arm SystemReady-compliant — meaning it can absorb a standard Arm software platform as-is.

The chip is built on a 2nm process, packing 11 IBM Z high-performance cores running at 5.7GHz+, SMT=2, with a 36MB private L2 per core that aggregates into a 432MB virtual L3 and 3.5GB virtual L4 cache. A dedicated on-chip DPU handles I/O acceleration. The availability target is 99.999999%.

Second-gen AI inference chip: redundancy as the differentiator

Most of the AI attention, however, is on the accelerator IBM disclosed alongside the CPU. Positioned as "enterprise GenAI," its spec sheet is unusually aggressive: 16 AI cores plus 1 redundant core (any single core failure is survivable), native FP4 and MXFP4 datatypes with TOPS up to 4x the previous generation, 96GB of HBM3e at roughly 4TB/s peak bandwidth (about 20x the prior generation), and PCIe Gen6 as a peer-to-peer low-latency host interface.

The "redundant AI core" choice is itself a tell. IBM is not positioning this part against training cards or general-purpose GPUs. It is built for the kind of AI inference that cannot fail: bank transactions, payment clearing, insurance adjudication. Like the rest of the Z stack, the accelerator is wrapped in confidential computing and quantum-safe cryptography, so data and models are protected at rest, in transit, and in use.

On the software side, IBM exposes these accelerators as standard platform devices under Linux on Arm, with latency comparable to native z-instructions; on z/OS they remain first-class instructions. That means a single mainframe can keep running COBOL and s390x workloads for legacy customers while new AI applications sit on top of the Arm software stack. KVM and OpenShift Virtualization fold both worlds into the same logical partition, with thread-level switching reported at nanosecond scale.

Why AI practitioners should care

Set this next to OpenAI's Jalapeño and NVIDIA's Vera, and the design space looks more interesting than usual. OpenAI is optimizing perf/W for its own model volume; NVIDIA is building a CPU-plus-GPU combo for general-purpose data centers; IBM is betting on a different axis entirely — making AI inference a native peripheral of the mainframe, and selling "enterprise-grade reliability plus confidential computing plus quantum-safe" as the killer feature.

Underneath, this is a bet that for genuinely mission-critical financial, medical, and government transaction loads, customers will pay not for "how many times faster" but for "does it stay up, can it survive an audit, and does it coexist with the legacy stack on the same box." If IBM's bet lands, the AI inference world will bifurcate: an "internet-grade" track tuned by OpenAI and NVIDIA, and a "mission-critical-grade" track where IBM sets the rules.

A cooler reading is this: roughly 70% of global transaction volume still runs on IBM Z. The dual-ISA-plus-AI-accelerator combo is essentially IBM's plan to keep that sixty-year-old franchise alive — bringing the Arm software ecosystem into the mainframe and extending the mainframe reliability moat into AI inference, with each side taking what the other needs. Whether the third path — beyond "x86+GPU" and "Arm server+discrete accelerator" — actually holds, will depend on what the first wave of customers measure in production.

So what

If you are choosing an AI inference stack today, IBM's new offering will not affect your decision in the short term — it targets private deployments with strict compliance, redundancy, and legacy coexistence requirements rather than large-scale cloud inference. But it sends a clear signal: as large-model inference runs into the walls of power, reliability, and compliance, bolting AI compute onto "mission-critical hardware" may run in parallel with the "stack more GPUs" playbook rather than competing with it head-on.

References: