Google puts encrypted inference on the table: HEIR compiler lets pretrained networks run on ciphertext
On August 14, Google published HEIR (Homomorphic Encryption Intermediate Representation) on its official security blog — the first open-source compiler aimed squarely at AI inference. Its goal: take any pretrained model and automatically convert it into a version that runs on encrypted data, so the cloud server never sees the plaintext at any point in the pipeline. The author is Jeremy Kun, Staff Software Engineer at Google. The compiler itself is open-sourced under google/heir on GitHub.
What HEIR actually fixes
To get a neural network running on ciphertext used to mean months of manual rewriting by a team of cryptographers. HEIR turns that into a "one-click" operation: write your program in Python, annotate which fields are confidential, and HEIR compiles the encrypted version for you. In other words, it lifts FHE from a cryptographer's craft project to an off-the-shelf engineering component.
Four reference applications that already work
Google did not stop at the compiler. It also released four real-world models, all compiled and verified:
- Deep Learning Recommendation Model — joint work with Belfort Labs, LG, and NYU. Service providers can recommend content without seeing any user feature vector.
- Credit card fraud detection — with Niobium and hardshell.ai. Banks can encrypt transactions and send them to a third-party fraud model, leaving only ciphertext exposed.
- Network intrusion detection (Kitsune) — with Niobium. The system can flag anomalies on encrypted network traffic without ever seeing packet contents.
- Hotword detection model — with Belfort Labs. An audio-triggered AI agent can recognize "Hey Siri"-style wake words without learning what was actually said.
All four were run on a single-threaded CPU, with latency numbers published. Google is explicit that this is a starting point — the company is working with Belfort, Niobium, Cornami, and Optalysys on dedicated FHE hardware accelerators that will push latency down further.
Why this matters more than it looks
Homomorphic encryption for AI has long been "theoretically perfect, practically unused". The capability is seductive — the server computes on ciphertext and returns ciphertext; only the user can decrypt — but the performance cost and migration burden have kept it locked inside research papers. HEIR is designed to push that cost curve down.
Google publicly lists its academic collaboration network: Georgia Tech, CMU, UCSB, Illinois Institute of Technology, Purdue, the University of Edinburgh, and Tsinghua University. Four peer-reviewed publications are already built on top of HEIR. That means HEIR is no longer just a Google tool; it is becoming a shared substrate for the whole research community.
What this means for you
For AI engineers: expect "private inference" to become a distinct product category in the next 6–12 months. HEIR makes it possible for the first time to answer both "I don't want to ship my model weights to the cloud" and "the user doesn't want their data seen by the cloud" with a single stack.
For enterprises: healthcare, finance, and government — the sectors most constrained by compliance — used to be stuck choosing between on-prem deployment (expensive) and federated learning (architecturally complex). Encrypted inference is a third path: put the model in the cloud, but let the server know nothing about the inputs.
For individuals: your next AI assistant may not need to know what you said. It can complete tasks on ciphertext and still help you.
For the industry: while Anthropic, OpenAI, and Google push "memory and personalization", encrypted inference may become a counter-position — "we don't read your data, and we still serve you well".
What is still missing
Google itself is clear that HEIR is not a silver bullet. The current latency overhead is still meaningful, and "product-grade" deployment still depends on the hardware accelerator roadmap. But the direction is set: over the next few years, AI inference will be tightly coupled with encryption rather than running almost naked, as it does today. HEIR is the first serious open-source attempt that turns that direction into something you can actually try today.
Closing
The story of FHE on AI inference has been told for years; what it was missing was a tool that broke the engineering barrier. HEIR, backed by Google's four reference applications and a hardware partnership matrix, is the first time that story has had a "you can pick it up and try it now" shape.
Privacy and AI are not opposing ends. HEIR's existence is itself proof — you can let a powerful model run on what it cannot read, and that capability is becoming engineering-grade rather than PowerPoint-grade.
Sources: Google Blog (blog.google/security/how-google-is-making-private-ai-practical-with-homomorphic-encryption/), AI Weekly, Google HEIR project page (heir.dev).