The leak is the ordinary working day
When staff paste company data into public AI tools, that data can train the provider's models and resurface in other users' answers. No attacker is required.
Four pieces of evidence, all third-party and sourced:
- At the New South Wales Reconstruction Authority, a spreadsheet with names, addresses, dates of birth and sensitive details was uploaded to ChatGPT in March 2025. The authority counts 2,031 people affected (nsw.gov.au).
- Cyberhaven Labs measures that 39.7 % of inputs to AI tools contain sensitive data (cyberhaven.com).
- IBM puts an AI-enabled breach at 6 million US dollars on average (ibm.com).
- The European Commission is enforcing the AI Act (digital-strategy.ec.europa.eu).
Five places where it happens
- PII in employee prompts. Names, addresses, IDs, often pasted unthinkingly.
- Trade secrets in "draft me something". Roadmaps, pricing, M&A drafts as context for a public model.
- Patient IDs to external APIs. Clinical workflows with no masking before egress.
- Source code uploads. Developers debug with an assistant; the IP leaves the building.
- Contract clauses to summarizers. NDAs and master agreements become training data for someone else's model.
What actually helps
A block moves the leak. What works is a control layer in front of the model: it classifies every prompt in real time, replaces sensitive entities with stable tokens, decides per data class between allow, mask, block and process locally, and logs the decision.
What this does not do. It inspects what goes into the AI, not what a human does with the answer afterwards. And the channel is AI use, not email, USB or a photographed screen.