Shipping LLM Features Without Leaking Customer Data: The Architecture That Holds Up

Where customer data actually flows in an LLM system — logs, embeddings, vendor APIs, eval sets — and the isolation, redaction, and retention choices that pass a real security review. With a pre-launch checklist.

A code review catches the obvious mistake: the API key in the frontend bundle. A security review catches the one that ships anyway — the support transcript that landed in a third-party trace tool, the embedding of a medical record sitting in a vector index nobody put access controls on. LLM features leak customer data in places that don't look like data flows.

The model call is the part everyone worries about and the part that's easiest to lock down. The leaks come from everything around it: the logs you added to debug a bad response, the eval set you froze six months ago, the vector store you spun up in an afternoon. Here is where customer data actually travels in an LLM system, and the choices that hold up when someone with a checklist starts asking questions.

Map the flow before you defend it

Every LLM feature has a data path wider than "user types, model answers." Trace every hop before you decide what to protect. A typical request touches more systems than the diagram in your head.

• The prompt, assembled from user input plus retrieved context
• The vendor API and every sub-processor behind it
• Observability and tracing tools
• The vector store holding your embeddings
• Eval and fine-tuning datasets
• Prompt and response caches

Draw the trust boundary. Anything that crosses it — leaving your VPC or your cloud account — is a decision that needs a reason, not a default you inherited from a tutorial.

The vendor API is the least of your problems

Providers made this part easy. Zero-data-retention endpoints, a signed DPA, and no training on your data on business tiers. Anthropic and OpenAI both offer ZDR configurations. Get the DPA, read the sub-processor list, and confirm retention is set to zero on the endpoints you actually call.

Be honest about the effort here: this is table stakes, solved mostly by contract and configuration. Self-hosting a model to avoid the vendor entirely is usually the wrong trade — you take on GPU operations, accept worse quality, and you still own every leak below. It only earns its cost under hard data-residency mandates, not general nervousness.

Redact before the token crosses the boundary

This is the cheapest control with the biggest payoff. Run PII detection on the prompt before it leaves your perimeter — Presidio, a cloud DLP API, or a NER model. Replace matches with placeholders and, if the response needs the real values, restore them on the way back. Reversible tokenization keeps the mapping inside your walls and sends the vendor a stripped prompt.

• Names, emails, phone numbers
• National IDs and passport numbers
• Card and account numbers
• Free-text medical or financial detail

Do not over-redact. Strip the account number the model needs to reason about and the feature quietly breaks. Remove what has no bearing on the task; keep what the task requires and protect it downstream instead. Tune for high recall on the severe fields and accept some false positives — a redacted street name costs nothing, a leaked one costs a disclosure.

Your logs leak more than your model calls

This is the real breach vector. Teams pipe full prompts and completions into LangSmith, Langfuse, or Datadog to debug a bad answer. Now customer PII lives in a third-party SaaS with different retention, different access controls, and a different auditor than your primary database.

• Log request IDs and metadata by default, not raw content
• Sample raw payloads behind a flag with a short TTL
• Scrub before shipping anything to an external tool
• Put separate, tighter access controls on trace data

Default retention on trace tools tends to run 30 to 90 days. That is up to three months of plaintext customer data sitting in a system your data-protection officer never reviewed, accumulating quietly with every deploy.

Embeddings are customer data, not math

The common misconception is that a vector is anonymized. Embedding inversion attacks can reconstruct meaningful pieces of the source text from the vector alone. Treat the vector store as if it holds the underlying content, because for practical purposes it does.

Then there is multitenancy. The classic RAG leak is one tenant retrieving another tenant's documents through a shared index. Enforce isolation server-side — a namespace or collection per tenant, or a metadata filter applied by your backend. Never trust a filter set by the client.

• Per-tenant namespace, or a mandatory metadata filter enforced server-side
• Encryption at rest
• The same access model as the source documents
• Delete embeddings when the source record is deleted

A separate index per tenant is the cleanest isolation, but it does not scale to thousands of small tenants. Metadata filtering with enforced, RLS-style checks is the pragmatic middle — as long as the check lives in code the client can't reach.

Eval sets and fine-tuning are the copies you forget

To measure quality you freeze real traffic into an eval set. That is a static copy of customer data, sitting outside your primary store — often in a git repo or an object bucket, often with real PII, often readable by the entire team. It escapes every control you put on the live database.

• Redact or synthesize eval data; keep only a small real holdout under tight access
• Version and access-control eval sets like production data
• Apply the same discipline to fine-tuning corpora
• Track lineage so a deletion request can reach these copies

Fine-tuning is worse: it bakes the data into the weights. Right-to-erasure can't be honored by deleting a row — you would have to retrain. Don't fine-tune on raw PII. Use redacted or synthetic data, and keep the mapping of what went into which checkpoint.

Deletion that survives an audit

Right-to-erasure and DSAR requests are where the architecture gets tested for real. A delete has to reach every copy: primary database, vector store, trace logs, caches, eval sets, and backups. Miss one and the deletion is a promise, not a fact.

• A data inventory mapping each store to a deletion path and a TTL
• Propagation of deletes to embeddings and caches, not just the source row
• Short TTLs so logs and traces expire on their own
• Documented lineage for every frozen dataset

If you can't answer "when this customer asks to be forgotten, what deletes and how" for every store, you don't have a system that passes review. You have one that hasn't been reviewed yet.

A checklist you can hold in your head

Before shipping an LLM feature, walk the data. For each place it lands, ask one question: is it here because the feature needs it, or because a default put it here. Then redact at the boundary, isolate per tenant, log metadata instead of content, and confirm every store has a delete path. The features that pass a security review aren't the ones with the best model. They're the ones where someone traced the data end to end before the auditor did.