Google Just Made AI on Encrypted Data Practical

For thirty years, homomorphic encryption was the holy grail of cryptography: compute on data while it stays encrypted. The theory was sound. The practice was that you could encrypt a million numbers and it would take three weeks to add them.

Google just shipped a compiler that changes that equation. Not for all workloads. For the ones that actually matter.

Google released HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler toolchain that can convert pre-trained AI models, ones that normally operate on unencrypted data, to instead operate on encrypted inputs. The server runs your model on encrypted data. Never decrypts. Never sees what it's computing on. Your data stays private.

The tool is designed to make homomorphic encryption, a technique that allows computations to be performed directly on ciphertexts, accessible to people who are not expert cryptographers.

This isn't vaporware. This is infrastructure you can use today. But you need to understand what it solves and what it doesn't.

The Problem It Actually Solves

Let's be precise about what's broken with current privacy approaches.

You have data you don't want to leak. Credit history. Medical records. Financial transactions. Behavioral signals that reveal who you are.

Option 1: Don't send it anywhere. Process locally. Problem: your device doesn't have the compute power or the model sophistication of a data center. Spam detection, fraud scoring, personalization—all require server-side infrastructure.

Option 2: Send it encrypted. Keep the encryption key private. Problem: the server still needs to decrypt to process. The moment decryption happens, your data is exposed to breaches, legal discovery, regulatory violation, or an engineer with access who shouldn't have it.

Option 3: Trust the server. Sign a contract. Hope their security is better than yesterday's. Problem: this is what everyone does now, and it fails constantly.

Homomorphic encryption offers Option 4: send encrypted data, server processes it encrypted, you get the answer back without the server ever seeing the plaintext.

A solution to these issues is homomorphic encryption, a rapidly maturing technology that fundamentally alters this trade-off by allowing computations to be performed directly on encrypted data.

That's not hyperbole. That's the actual inflection point.

Why This Matters for Three Specific Industries

Credit Scoring & Fraud Detection

A credit card company scores a transaction for fraud risk in milliseconds. That score depends on your entire transaction history, your location, your typical purchase patterns—all private data.

Today, the score runs on your data in plaintext. The model sees everything. An API integrates with third-party services (some of which are less secure than you'd like). If any system is breached, your financial history is exposed.

With HEIR, the credit scoring model runs on encrypted transaction data. The company gets the fraud score. Never decrypts your history. The server can't see what it's processing.

That's cryptographically guaranteed, not contractually promised.

Healthcare & Medical Records

A doctor wants to run your medical records through a disease prediction model. The model is proprietary—the hospital can't give you a copy because it's valuable IP.

You also can't send your unencrypted records to the hospital's server if you don't trust it (or their security).

HEIR lets the hospital run the encrypted inference. The model runs on your encrypted medical history. The hospital gets the prediction. Your data stays encrypted the entire time.

Real use case that's blocked today: cross-hospital analysis. Hospital A wants to pool anonymized data with Hospital B to train a better model. HIPAA prohibits it. With FHE, the analysis runs on encrypted data from both hospitals. Neither hospital decrypts the other's data. The output is insights, not raw data.

Recommendations & Personalization

You know personalization requires the server to know everything about you. Your watch history, your reading history, your search history, your purchase patterns.

With HEIR, the recommendation model runs on your encrypted profile. The service returns personalized recommendations. Your behavior stays encrypted.

Standard protections like end-to-end encryption present a trade-off: user-data can be protected from data breaches, but then the service provider cannot provide features that depend on the data, such as spam or virus detection. HEIR removes that trade-off for narrow, high-value workloads.

Why Nobody's Using This Yet (The Honest Version)

FHE runs on iPhones and Ethereum, yet stays 1,000x slower than plaintext.

That's the gap. Your encrypted fraud model doesn't run in milliseconds. It runs in seconds or minutes. That might be acceptable for batch scoring (do fraud scoring every hour). It's not acceptable for real-time credit authorization.

For private LLM inference today, confidential GPUs are the pragmatic option. A Trusted Execution Environment (TEE) like Intel TDX or a confidential GPU runs your model at near-native speed while isolating the computation from the OS and other processes. No thousand-fold overhead. You just trust that the CPU vendor built the security right and there are no side-channel attacks.

FHE removes that trust entirely. The tradeoff: you pay 1,000x in performance but gain mathematical certainty that the server can't see your data.

A TEE runs plaintext computation inside hardware isolation at near-native speed, but you trust the chip vendor and the absence of side-channel attacks. FHE removes that hardware trust entirely at a cost of three to four orders of magnitude in performance.

The choice depends on your threat model. If you're worried about a nation-state with access to Intel internals, FHE is the right choice. If you're worried about your cloud provider's sales team, a TEE is probably enough.

What HEIR Actually Changes

HEIR is not a breakthrough in cryptography. Fully homomorphic encryption has existed since Gentry's 2009 paper. The breakthrough is in usability.

HEIR serves as a powerful development platform and compiler toolchain that automates the conversion of existing models into FHE-compatible versions. By supporting multiple FHE schemes and high-performance backends, HEIR ensures your encrypted workloads run with optimal efficiency. Leveraging MLIR (Multi-level Intermediate Representation), HEIR provides the abstraction necessary to represent and scale complex models across diverse dialects.

Translation: You don't need to be a cryptographer. You write a model in PyTorch, feed it to HEIR, and get back an FHE-compatible version. The compiler handles the boring parts (scheme selection, bootstrapping, circuit optimization).

Google's vision is to make HEIR a one-click solution to enable non-experts to incorporate encrypted inference into production applications.

That's the real shift. Not the cryptography. The tooling.

Where This Wins (And Where It Doesn't)

Wins:

• Batch processing with latency tolerance (fraud scoring, compliance checks, monthly recommendations)
• Small models or specific layers (classification, regression, decision trees)
• High-value data that justifies overhead (medical, financial, behavioral)
• Lookups and exact matching (does this transaction match a blacklist?)

Doesn't win:

• Real-time inference on large models (LLMs, transformer attention is expensive encrypted)
• Interactive computation where latency kills UX
• Scenarios where TEE is sufficient (you mostly trust the vendor)
• Cost-sensitive applications (1,000x overhead is expensive)

For narrow workloads: private lookups (the Apple and Microsoft pattern), small-model ML inference, and encrypted logic at tens of transactions per second, FHE is practical. No, for general-purpose or interactive computation, where the 1,000x to 10,000x overhead still rules it out.

The Market Signal

OpenFHE is the reference open-source library for CKKS and BGV, Apple open-sourced its BFV stack, Google develops the HEIR compiler, and Duality and Tune Insight serve enterprise analytics. FHE in 2026 is neither grail nor vaporware.

Apple open-sourced an FHE stack. Google shipped a compiler. Zama (TFHE) hit unicorn status in 2025. Duality and Tune Insight are building enterprise products on top.

This is no longer "cool research." This is infrastructure getting built.

The Real Consequence

The consequence is that privacy isn't a binary choice between "send encrypted and useless" or "send plaintext and vulnerable."

For workloads that can tolerate latency, you can have both: process encrypted data, get the answer, never leak the input.

That changes threat modeling for regulated industries (healthcare, finance, government). If you're building a service that touches sensitive data, the question is no longer "how do we secure the server?" It's "which operations can run encrypted, and which ones need plaintext?"

Organizations and governments around the world continue to explore how to use PETs (Privacy-Enhancing Technologies) to tackle societal challenges and help developers and researchers securely process and protect user data and privacy.

That's bureaucratic language for: we're betting on this. Governments are exploring FHE for census data, tax records, medical registries. Not because it's magical. Because the alternative—trusting servers with plaintext—keeps failing.

The Takeaway

• Homomorphic encryption is no longer theory. Google shipped production-ready tooling. HEIR converts existing models to run on encrypted data without requiring cryptography expertise.

• The throughput cost is real: 1,000x slower than plaintext. That's acceptable for batch scoring (fraud, compliance, recommendations). It's not acceptable for interactive inference.

• For high-value data (medical, financial, behavioral), the overhead might be worth it. Especially if regulatory compliance or data trust is your constraint.

• TEE (Trusted Execution Environments) still wins for most workloads. Encrypt once, run plaintext inside hardware isolation, trust the chip. Cheaper and faster than FHE.

• The market is sorting into two camps: TEE for most work, FHE for high-trust scenarios. Not one replacing the other. Different tools for different threat models.

• Open-source tooling is maturing fast. HEIR, OpenFHE, Zama. You no longer need to hire cryptographers to explore this.

• Privacy isn't binary anymore. You can process encrypted data, get answers, never see plaintext. For workloads that matter, that's a fundamental shift.

The catch: you need to know which workloads those are. HEIR makes encrypted inference accessible. It doesn't make it free. The question for your team isn't "should we use FHE?" It's "which of our operations justify 1,000x overhead for mathematical privacy guarantees?"

Answer that correctly, and HEIR is a one-click solution. Answer it wrong, and you've added latency for no gain.

Google just gave you the tool. Now you need to decide when to use it.

---

Tags: privacy, homomorphic-encryption, security, fhe, machine-learning, data-protection