Architecture guide

TEE-Terminated TLS vs E2EE

A plain-language guide to confidential AI inference. The most important question is: where does your prompt first become plaintext?

Based on primary protocol and attestation standards

Direct answer

What is the difference between TEE-Terminated TLS and E2EE?

Both designs can keep the prompt plaintext inside an attested TEE. With TEE-terminated TLS, the TEE holds the HTTPS private key and the gateway only forwards encrypted traffic. With application E2EE, the gateway may decrypt outer HTTPS, but the app has already encrypted the AI payload to a separate key bound to the TEE.

A TEE-only gateway is different: it decrypts the prompt before forwarding it to the TEE, so the gateway can read it. Think of the TEE as a protected room, attestation as its signed inspection report, and encryption as the locked delivery path into that room.

01

Where does encryption end?

Find the first component that holds a decryption key and can see the prompt.

02

Is that key attested?

Verify that the decryption key belongs to the intended TEE, not merely to the provider.

03

Did this request use it?

Look for fresh, request-bound route evidence rather than an unrelated backend quote.

Three architectures

Where the prompt first becomes plaintext

All three designs may advertise HTTPS and a TEE. Their privacy guarantees differ because the decryption key lives in a different place.

A

Direct attested channel

TEE-Terminated TLS

Your app opens a normal HTTPS connection, but the TLS private key is held inside the attested TEE. Load balancers may forward encrypted traffic; they do not decrypt it.

The client or an independent verifier must check a cryptographic binding between the attestation report and the key used by the TLS endpoint. A normal browser padlock proves the domain name, not that the key is inside a TEE.

Your app Prompt encrypted by TLS
Network / load balancer Forwards ciphertext and sees metadata
Attested TEE TLS key, first plaintext, model execution
Gateway access
Cannot read the prompt if TLS is passed through
Client change
Usually standard HTTPS plus attestation verification
Critical proof
TLS endpoint key is bound to a fresh valid quote
B

Payload protected through the gateway

Attested Application E2EE

Your app first verifies a public encryption key from the attested TEE. It then encrypts the prompt to that key before sending the request through the ordinary HTTPS API.

The gateway may terminate the outer TLS connection, authenticate the account, bill the request, and route it. It sees ciphertext instead of the protected prompt. The response must receive equivalent protection. If the app accepts an unverified key supplied by the gateway, the gateway can substitute its own key and the E2EE claim fails.

Your app Verifies attestation, encrypts payload
HTTPS API gateway TLS ends; E2EE payload stays encrypted
Attested TEE E2EE key, first plaintext, model execution
Gateway access
Sees routing metadata and ciphertext, not protected content
Client change
Requires an SDK or application encryption protocol
Critical proof
E2EE key, freshness, request, and response are bound correctly
C

Backend protection only

TEE-only Gateway

Your HTTPS connection ends at a conventional API gateway. The gateway decrypts the request and then sends it to a TEE backend, often over another protected connection.

The gateway can read the prompt. This design can still protect model execution from the backend host or cloud administrator, but it does not protect prompts from the API operator, gateway logs, or a compromised gateway.

Your app Prompt encrypted by TLS
HTTPS API gateway TLS key and first plaintext
TEE backend Receives prompt for model execution
Gateway access
Can read and potentially log the prompt and response
Client change
Works with ordinary HTTPS clients
Critical limitation
No cryptographic client-to-TEE content boundary

Side-by-side

TEE TLS vs E2EE: pros and cons

E2EE is not automatically better. The right design depends on the threat and the quality of its implementation.

Question TEE-Terminated TLS Attested Application E2EE TEE-only Gateway
First plaintext Inside the attested TEE Inside the attested TEE At the API gateway
Gateway sees content No, when it only passes TLS through No, for fields inside the E2EE envelope Yes
Client integration Standard HTTPS plus attestation checks Encryption SDK or protocol integration Standard HTTPS
Attested key TLS endpoint key Application encryption key Usually only a backend identity
Operational strength Streaming and API compatibility Works through TLS-terminating gateways and CDNs Simple routing and compatibility
Main tradeoff TEE must manage TLS identity and direct connections More client, key, retry, and streaming complexity Gateway remains trusted with plaintext
Route proof Strong when the live TLS peer key is attested Strong when requests and responses bind to the attested key A separate backend quote is not enough

Threat model

Who can still see or influence a request?

A threat model names the people and systems you do not want to trust. No single badge covers every threat, so this directory treats hardware, channel, workload, model, and route evidence separately.

Network observer

Protected by all three designs. TLS hides content while it crosses the public network. IP addresses, endpoints, and traffic patterns can remain observable.

Cloud or host administrator

TEE protection is intended for this threat. A valid, policy-approved quote can show that execution is isolated from the host, subject to hardware, firmware, and side-channel limitations.

Gateway or CDN operator

Protected only when the content key stays beyond the gateway. TEE-TLS requires passthrough. Application E2EE keeps the payload encrypted even when outer TLS ends there.

Malicious code inside the TEE

Not solved by encryption or isolation. Code that legitimately receives plaintext can log or export it. Measurements and source provenance tell you what ran; review and policy determine whether that code is acceptable.

Reseller or compromised control plane

A backend quote alone is insufficient. The evidence must connect the request to the claimed model and backend. Otherwise a valid enclave can exist while traffic takes another route.

Metadata observer

Traffic size and timing normally remain visible. Account identity, billing, IP address, model route, and unencrypted headers may also be exposed even when the prompt is protected.

The reseller problem

Infrastructure evidence is not request evidence

The API provider owns the endpoint and customer relationship. The infrastructure provider operates the TEE, GPU, or managed confidential platform. An aggregator may use several infrastructure providers, even for models sold under one brand.

A strong request-bound route connects a fresh client challenge, requested model, backend identity, and response to the same attested key. Without that connection, attestation proves that a suitable backend exists, not that it served this request.

Choosing a design

Start with the operator you need to exclude

I need standard API compatibility

Prefer TEE-terminated TLS when the service can prove the live TLS key is inside the TEE. It usually preserves normal clients, streaming, retries, and tool calls.

I must use an external gateway or CDN

Prefer attested application E2EE when that intermediary must route requests but must not read prompts. Confirm both request and response protection.

I only need backend host isolation

TEE-only may be sufficient if you explicitly trust the API operator and gateway with plaintext. It should not be described as protection from that operator.

For either strong channel design, verify:

  • A fresh hardware quote validates against the vendor trust chain.
  • The decryption key is cryptographically bound to that quote.
  • The measured workload matches a reviewed policy and known source or build.
  • The requested model and actual route are bound to the response where possible.
  • Failures stop the request instead of silently falling back to an unverified path.

Common questions

Confidential AI inference FAQ

Direct answers to the terms and limits shown throughout the directory. Each heading has a stable URL that can be cited or shared.

What is confidential AI inference?

Confidential AI inference means running an AI model while limiting which surrounding systems and operators can read the prompt, response, or model data. It usually combines a TEE, remote attestation, and an encrypted channel. The actual guarantee depends on where that channel ends and what the attested software does.

What is a trusted execution environment (TEE)?

A TEE is an isolated area of a processor or accelerator. It is designed to keep the data and code inside it hidden from the host operating system, hypervisor, and cloud administrator. It reduces who must be trusted, but it is not a guarantee that the application itself is safe.

What is remote attestation?

Remote attestation is a signed hardware report. It can identify the TEE platform, security state, and a measurement of the software that started inside it. A verifier checks that report against the hardware vendor's root of trust and an explicit policy.

What does attestation not prove?

A valid quote does not prove that the measured code is well designed, that it has no logging or export feature, that it loaded the advertised model, or that your request reached it. Those claims need additional evidence such as reviewed source, reproducible builds, model measurements, key binding, and request-bound route evidence.

What does TLS termination mean?

TLS termination is the point where an HTTPS connection is decrypted. Before that point, intermediaries can forward encrypted bytes but cannot read the prompt. After that point, the component holding the TLS private key can read it. The security question is therefore not whether HTTPS is present, but where its private key lives.

What does end-to-end encryption mean here?

On this site, E2EE means that the client encrypts the AI payload to a key proven to belong to an attested TEE. An outer HTTPS connection can still end at an API gateway, but the gateway receives an encrypted prompt and cannot decrypt the protected payload.

Is E2EE always better than TEE-terminated TLS?

No. Both can create a cryptographic channel from the client to an attested workload. TEE-terminated TLS is easier for standard HTTPS clients and streaming. Application E2EE can work through a gateway or CDN that terminates TLS, but it needs client-side encryption, key verification, replay protection, and careful response handling.

Why is TLS still needed when the prompt uses E2EE?

TLS protects the outer API connection, credentials, headers, and protocol behavior from network attackers. Application E2EE adds a second protection boundary for the prompt and response. The two layers solve different parts of the problem.

What is a TEE-only gateway architecture?

The public gateway terminates HTTPS and sees the plaintext prompt, then forwards the request to a TEE backend. The TEE can still protect model execution from the backend host or cloud operator, but the gateway and anyone able to control it remain in the prompt trust boundary.

Does a verified backend prove that my request used it?

Not by itself. A provider or reseller can show a valid quote from one backend while routing a request somewhere else. Strong route evidence binds a fresh client challenge, requested model, backend identity, and response to the same attested key or signed receipt.

Can attestation identify a reseller and the real infrastructure provider?

It can contribute evidence, but the roles must be recorded separately. The API operator is the company you call and pay. The infrastructure operator runs the TEE or managed platform. A reseller may route different models to different infrastructure providers, so attribution should be per model route rather than assumed from the brand name.

What information can remain visible?

Encryption normally does not hide IP addresses, account identity, billing events, request timing, traffic size, or every header. Model names and tool metadata may also remain visible when they are needed for routing. Hiding those signals requires additional protocol and network measures.

Can code inside the TEE log or leak prompts?

Yes. A TEE protects code and data from the surrounding host; it does not make that code trustworthy. Malicious or flawed code inside the TEE can log, return, or transmit plaintext. Software measurements, source provenance, reproducible builds, egress controls, and an understandable policy are therefore important.

Why can hardware pass while the overall result is partial?

Hardware verification answers only part of the question. We mark a check partial when the hardware or channel evidence passes but another link, such as image provenance, source mapping, model identity, freshness, or request routing, is missing. Partial means useful evidence exists, not that the complete end-to-end claim is proven.

Apply the guide

Check the claims against live evidence

Use these definitions while comparing a provider, a model route, or a current attestation result.

Primary references

Standards behind the terms

These sources define the underlying protocols and attestation architecture. Product claims still require provider-specific evidence.