The Data You Never Collect Cannot Leak

September archive edition, newly written October 10, 2026. The archive placement is editorial, not a claim of prior publication.

A support transcript becomes a prompt. The prompt becomes a saved conversation. The conversation becomes an evaluation example. A screenshot lands in a presentation. Nobody deliberately built a new customer-data system, yet one now exists across tools that have different access rules and retention policies.

Privacy-by-design starts before collection: identify the purpose, use the minimum information necessary, control who can access it, and know how every copy is retired. AI does not remove these obligations. It makes careless copying faster.

The customer’s context matters more than your convenience

People disclose information to get something done. A customer may share financial trouble with support, a patient may describe a symptom, or a job applicant may explain a personal circumstance. They are not necessarily volunteering that material for every future analytical workflow.

The trust question is not merely “Would this leak embarrass us?” It is “Would this person reasonably expect this use?” A technically secured workflow can still surprise or disadvantage the people represented in its data.

NIST’s Privacy Framework offers a voluntary way to reason about privacy risk. NIST’s AI Risk Management Framework organizes AI risk work around Govern, Map, Measure and Manage. Neither is a substitute for the law, a vendor contract or an actual implementation review.

Purpose comes before the prompt

Write down the decision the workflow supports. “Improve customer service” is too broad. “Find common confusion about the cancellation policy in approved, minimized support excerpts” is more actionable. The narrower purpose helps determine what information is actually needed.

You may need the customer’s question and the agent’s response, but not a name, street address, account number or payment detail. Removing direct identifiers helps, yet context can still identify a person. A rare event, a location and a date can be revealing together. Pseudonymization is not the same as anonymity.

Trace the copies, not just the source

Make an inventory of where data travels: source system, extraction script, temporary file, model request, conversation history, logs, evaluation set and analytics export. For each location, identify the owner, access policy, retention window and deletion mechanism.

Provider behavior varies by product, plan, configuration and agreement. Do not assume a consumer chat tool and an enterprise API have identical training, logging or retention terms. Verify the exact service you intend to use, including third-party tools an agent can call.

Retention is an operating process, not a sentence in a policy

A shorter window reduces the standing inventory in the simple model above. It does not automatically remove backups, exported spreadsheets or data already incorporated into another system. A deletion promise is only as good as its reach across the copy chain.

Define why information must persist and for how long. Then verify that deletion actually happens. Legal, contractual or operational requirements may require retention; minimization does not mean deleting evidence you are obliged to keep. Resolve conflicts with qualified reviewers.

Give agents permissions for the task, not the organization

Tool access is another boundary. An assistant that summarizes public help content does not need the customer database. A workflow that drafts a reply may not need permission to send it. Separate reading, proposing, approving and executing where consequences justify that separation.

Keep secrets out of prompts and client-visible files. Use approved credential handling and restrict access at the tool boundary. Instructions saying “do not reveal private data” are not a substitute for enforcing which data and tools the workflow can reach.

This connects directly to the writer becoming a website operator. Expanded capability must come with a narrower, explicit operating contract. Being able to move data is not authorization to do it.

A practical preflight before using real customer material

1. Purpose: state the specific job and the expected benefit.
2. Necessity: remove fields and context that are not needed.
3. Permission: verify the lawful basis, agreements and approved tools with the right owners.
4. Exposure: map who and what can access each copy.
5. Lifecycle: define retention, deletion, backups and incident response.
6. Evidence: rehearse on synthetic or approved public examples, then verify the controls before expanding.

Synthetic data is useful for rehearsing a workflow, but do not simply ask a model to “anonymize” a real transcript and assume the result is safe. Check what remains, where the original traveled, and whether the transformation served the stated purpose.

Trust is a constraint that enables better work

Privacy review can feel like friction when a promising tool is waiting. But a well-defined boundary lets a team move with fewer surprises. It makes the workflow easier to explain, audit, correct and hand to another owner.

For high-stakes workflows, bring legal, security and domain specialists in before production. For everyday work, start with a question that costs nothing: what can we accomplish without this information? The most effective protective control may be the copy that was never made.

Sources and further reading

• NIST — Privacy Framework
• NIST — AI Risk Management Framework

Continue across disciplines

• Accessibility Is a Product Decision, Not a Final Audit
• The Best Product Decision Might Be a Smaller Bet