You can encrypt a disk. You cannot yet encrypt a thought at the price of a normal ChatGPT reply. Apple's Private Cloud Compute is the closest production system. Zama's docs still put a GPT-2 token at about 11 seconds on GPU under fully homomorphic encryption, and about 300 seconds on CPU. OpenAI's zero data retention is a contract you apply for. Coverage uses one word, private, for all three.
Programmable trust already covered the cryptographic toolkit as a stack for markets. This is the same toolkit aimed at a prompt: the sentence you did not want a vendor to read.
Tap to enlargeOn the phone, or not at all
Apple's public line since 2024 has been simple. If the model fits on the iPhone, it stays on the iPhone. Apple Intelligence routes small tasks to the Neural Engine. The data never becomes a row in a vendor log. That is real privacy. It is also a size limit. When the request needs a bigger model, the phone has to send the prompt somewhere. The question is which box, and who can read it.
Google's Gemini Nano runs inside Android's AICore service. Prompts stay on the device. AICore is isolated from most other packages; Google calls this Restricted Package Binding, and says the allowed list changes only on a full Android OTA. It has no direct internet. Model downloads go through a companion Private Compute Services APK. Google says AICore does not keep the input or the output after the call. On the phone, Google has a cousin of stateless and of no privileged runtime access. Non-targetability is free, because the box is in your pocket. Verifiable transparency is not: AICore does not publish the binaries the way PCC does. Hard prompts still leave.
Qualcomm's on-device AI stack and a pile of 1B-to-8B open weights try the same trick on Android and PCs. Ollama and llama.cpp made local inference ordinary for people who will install a runtime. A 7B-to-70B model on a laptop will do useful work. It will not match a frontier lab on the hard prompt. Default is still a HTTPS POST to a lab.
End-to-end encryption, the kind Signal and WhatsApp use for messages, does not save you here. E2E works when the service does not need to read the payload. A model has to read the payload. That is the inference. Apple said this out loud in the PCC launch post: complete E2E is not an option if the cloud model must see the request. OpenAI has to read the prompt. A mail service does not.
The trusted box
What shipped is a Trusted Execution Environment. A TEE is a mode of a chip that encrypts memory, attests what code is running, and tries to keep even the landlord out. Intel SGX was the early product. Then came TDX. AMD has SEV-SNP. AWS has Nitro Enclaves. Azure has Confidential Computing. Google Cloud has the same product line. NVIDIA put confidential computing on Hopper H100 and then Blackwell. The NVIDIA CC docs are the operator manual. The GPU holds plaintext in HBM, which NVIDIA treats as hard to tap with everyday tools. The PCIe bus to the CPU is encrypted. NVIDIA measures that bounce-buffer traffic around 4 GB/s. Unified memory is extra-expensive. If your serving stack pages KV-cache through the CPU, you will feel it. If the model stays on the GPU, you may not. Attestation tells the client, before it sends the prompt, that the remote code hash matches a known build.
The root of trust for Private Cloud Compute is custom Apple silicon with a Secure Enclave, the same family of hardware used in iPhone, plus a hardened OS described in Apple Platform Security. A normal GPU VM is not that hardware. The 2024 post listed five rules.
- Stateless. Use the personal data only to answer. Delete it after the reply. No log, no debug copy.
- Enforceable. Do not depend on a TLS-terminating load balancer that might dump requests during an outage.
- No privileged runtime access. No SSH. No break-glass shell for SRE.
- Non-targetability. An attacker who owns one box should not be able to steer one named user's requests onto that box.
- Verifiable transparency. Publish the binaries. Let researchers check that production runs the same bits.
Tap to enlargeIn 2026 Apple extended PCC onto Google Cloud. The five rules stayed. The hardware changed: NVIDIA Confidential Computing, Intel TDX, Google's Titan chip. Apple said this was the first time those primitives had been wired into an end-to-end confidential inference pipeline at global scale. Binaries still published. Security Bounty still covers research-mode nodes. Apple still signs the software the iPhone will trust. AWS and Azure were not chosen. Financial terms were not disclosed. The rollout was a summer 2026 preview, in parallel with Apple-silicon PCC, not a migration.
PCC on Google Cloud is still Apple's code. Google does not get to read Siri. Apple is renting halls and GPUs under attestation. If you do not trust Apple's signing key, Google's hall does not help. If you trust the key, which hall it sits in is not supposed to matter.
OpenPcc, a June 2026 preprint, tried to make a PCC-like serving stack on commodity TDX plus H100, with composite attestation. They reported single-digit-percent extra TTFT on Llama-3 8B once attestation was cached. That is a research prototype. Apple's five rules are becoming a pattern other people can copy, badly or well, without Apple silicon.
Overhead is where the marketing splits. A 2024 Hopper study, arXiv:2409.03992, put throughput tax under 7% for typical LLM queries, and near zero for large models and long sequences, because compute drowns the encrypted I/O. The same paper's time-to-first-token tax sat near 19% on H100 for Llama-3.1-8B. PipeLLM measured the other end: vanilla confidential serving dropped throughput 52.8% on OPT-30B and 88.2% on OPT-66B, then a pipelined runtime cut that to under 19.6% across 13B to 175B. The 2024 IEEE Cloud paper said large pipelined models can run close to par. Prefill and decode differ. Batch-1 and serving differ. Hopper and Blackwell differ. NVIDIA's own July 2026 Blackwell post put confidential inference at "up to 98%" of baseline on an HGX B300 running Qwen 3.5 397B at FP8 under SGLang, with most cells under 8% throughput tax. That is a vendor stack with CC-specific patches, not a promise about a stock Hopper job. A July 2026 cloud study, arXiv:2607.19353, measured a single H100 80GB under Intel TDX on Google Cloud's a3-highgpu-1g machine type: average TTFT rose 21.8% on Mistral-7B v0.1 and 27.8% on Qwen3-30B-A3B; global token throughput fell 17.7% and 21.1%. The same paper notes Secure Boot was off in that run. Quote the 21.8% as a capacity number, not as a security proof. A hospital that budgets H100 seats from the Blackwell 98% slide will run out of prefill first.
SGX's history is the caveat. Foreshadow (L1 Terminal Fault), Spectre, and years of microcode. Intel patched, renamed, and moved the product from SGX enclaves toward TDX VMs. AMD SEV-SNP encrypts VM memory with a key the hypervisor should not hold. AWS Nitro's pitch is a stripped hypervisor and a separate enclave CPU. A TEE shrinks the attack surface. It does not make a break impossible. Apple's non-targetability rule tries to make a single-box break useless for hunting one person. That design does not prove anything against a state with a supply chain.
The contract that people call private
Most companies that say "we never train on your data" are not running PCC. They are running a processor agreement.
OpenAI's data controls are specific if you read the table. API customer content is not used for training by default. Abuse-monitoring logs exist for 30 days on many endpoints. Zero Data Retention is a separate control. You apply. OpenAI approves. The project data-retention API is how a company turns it on after that approval. Some endpoints are eligible (/v1/chat/completions, /v1/responses). Some are not (/v1/conversations, Assistants, vector stores, videos). ZDR forces store to false even if your request asks to store. Modified Abuse Monitoring is a middle rung. Eyes-off and safety-retention variants exist for people who still need a human review lane. Data residency, where it exists, is chosen at project creation and is hard to bolt on later.
Anthropic's commercial terms and privacy policy, as of the June 2025 text that vendor audits still quote, say Anthropic may not train on customer content from the API. Default deletion is about 30 days. ZDR is a signed add-on: prompts and outputs not stored at rest after the response. In 2026 Anthropic's Covered Models policy, as summarised from live docs in August, put Claude Fable 5 and Mythos 5 on a 30-day retention that is not available under ZDR. Read the current commercial terms before you rely on a model name.
Amazon Bedrock tells a cleaner story for the default: Bedrock does not store inputs and outputs, and model providers do not see prompts. Eligible customers can still ask for a stricter ZDR through the account team.
Google's live page is no longer titled "Vertex AI and your data." The old path 404s. The document that now holds the table is Gemini Enterprise Agent Platform and zero data retention. Training restriction is in the Service Specific Terms: Google will not train or fine-tune on your data without prior permission. Abuse-monitoring prompt logs exist for customers on the Google Cloud Platform Terms of Service; an exception is a request, not a default. Some Advanced AI features, under the Advanced AI Safety Addendum, log prompts and responses and may not be eligible for ZDR at all. Grounding with Google Search stores derived queries for up to three days, with no off switch. Grounding with Google Maps stores prompts and outputs for 30 days, with no off switch. The Interactions API defaults store to true. If you do not set store=false, Google keeps prompts, responses, and conversation state so you can retrieve the thread later. Google's in-memory cache keeps inputs and outputs for 24 hours at project level and, in Google's wording, "does not violate zero data retention" because it is not at rest. You can disable the cache. You cannot disable Search grounding and still use Search grounding. The Cloud Data Processing Addendum is the contract under those rows. Same noun as OpenAI's ZDR. Different disk. Different defaults. A hospital counsel who thinks ZDR means "no copy exists" will lose that fight. A copy exists. Google says that copy does not count as retention.
A deletion contract does not encrypt the GPU. A site reliability engineer with a lawful break-glass, a national order, or a bug can still see a live request on a normal GPU. ZDR means the disk after the call should be empty. It does not mean the RAM during the call is empty. HIPAA and GDPR Article 32 can make you demand the contract. They cannot encrypt the forward pass. Appropriate security, in 2026, is still usually TLS plus access control plus a DPA. A hospital can be HIPAA-aligned and still send plaintext prompts to a GPU in Virginia. The Act did not name TEEs. Buyers who need more have to write it into the contract: attestation, named builds, no subcontracted decode, deletion of KV-cache, no human review except on a listed channel. The Cloud Security Alliance and an AICPA SOC 2 letter are about the vendor's process. They are not an attestation quote.
Homomorphic encryption and proofs
Fully homomorphic encryption lets a server compute on ciphertext. Craig Gentry's 2009 thesis showed it was possible. For a decade it was a toy. Microsoft SEAL, HomomorphicEncryption.org, and Zama made compilers. Concrete ML's LLM path is a split: the client runs attention and activations in the clear on the client's machine; the server runs linear layers on encrypted data. Ciphertexts are about 4x the clear tensors. A Llama-3.2-1B model moves about 18MB per token. GPT-2 moves about 2.2MB. GPU helps by about 30x. 11 seconds per GPT-2 token is not a chat product. Zama's hybrid phi-1.5 experiment, on two metal boxes, hit about 2.5 seconds per token against about 50ms on-prem plaintext. Fifty times slower, and that was a small model with a client still doing real work.
Today FHE is useful for a linear layer you do not want to show the cloud, or a small model on a regulated field. It is not ChatGPT with encryption. If a vendor says FHE and also says frontier chat, ask for the milliseconds. The God Protocol already flagged that FHE stays orders of magnitude slower than plaintext. That sentence is still true in 2026 for anything a person would talk to.
zkML is a different tool. A zero-knowledge proof shows that a model produced an output from an input under a committed set of weights, without showing the weights. The feature is integrity of the output, not confidentiality of the prompt. EZKL compiles ONNX to Halo2. Their own benchmarks beat RISC Zero and Orion on small models. Lagrange's DeepProve is the 2025-2026 attempt to prove a full transformer. The IACR eprint, 2026/1112, dated May 2026, reports GPT-2 at about 174 tokens per minute and Gemma 3 at about 86, with verification in 1 to 3.7 seconds, and 1855 tokens per minute when proving is split across machines. That is a proof of the output sequence, not a private chat. GPT-3 scale is not in the paper. The practical pattern is: answer now, prove later, act on the proof if the money is large. Modulus and Giza chased on-chain verification for tiny networks. Useful for a price feed. Not a private diary.
FHE hides the data from the server. zkML proves the server ran the code. You may want both. Most buyers currently have neither.
Match the tool to the secret
If the prompt is the secret, you want on-device or a TEE with published code, or FHE if the model is small enough to wait. A ZDR contract is extra, not the core. If the weights are the secret, zkML is the version you can verify without seeing the file. Expect a wait. If the output is the secret, TLS already encrypts the channel home; ZDR and enterprise terms are for the vendor's log. If the identity is the secret, PCC's non-targetability is the relevant sentence. A normal API key plus an invoice is a name.
Ask for the artefact: an attestation quote, a binary hash, a retention table by endpoint, the named models excluded from ZDR, the last third-party audit, and the SGX-class caveats. A blog post that says never is not that list.
Differential privacy and federated learning get filed under private AI. They do not belong there. Differential privacy adds noise so a published statistic does not reveal a row. Apple's privacy overview describes that use for some device analytics. It does not hide your prompt from the model that must follow it. Federated learning trains a model across devices without pooling raw data. That is a training-time story. Inference still needs the prompt in the clear inside whatever box runs the weights. The Confidential Computing Consortium is the vendor club for TEEs. HomomorphicEncryption.org is the club for FHE.
A hospital that hears "private AI" and buys API ZDR has bought a deletion policy. A bank that hears "homomorphic" and expects ChatGPT-speed has bought a demo. A protocol that hears "zkML" and expects hidden user queries has bought a verifier. A consumer who hears "Apple Intelligence" and assumes every Siri-class request stays on the phone has not read the routing line. Small jobs stay on the phone. Large jobs go to PCC. The phone decides.
Google's ZDR table never uses the word private. During the call, the prompt is still in RAM.