In short: A generated summary must never be the only surviving copy of a decision. Every summary is a lossy compression made by one model on one day. Provenance is the asset. The summary is a view.
Key points
- Summaries often drop hedges, caveats, minority views or exact commitment wording that later disputes turn on.
- Store raw sources immutably with retrieval timestamp, origin and verifier.
- Version each summary with the model and prompt that produced it, and regenerate when models change.
- Handle deletion with tombstones that record what was removed and flag dependent records.
- Storage is cheap; reconstructing a decision from a summary nobody trusts is not.
The tempting design stores what is useful. Each interaction becomes a clean summary, tagged and embedded. The raw notes, transcripts and documents are discarded, or left in some other system with a different retention policy. Within a year the summaries are all that remain, and nobody decided that on purpose.
Why that is dangerous
Every summary is a compression made with a particular model, prompt and set of assumptions. It keeps what that model judged important and drops the rest. Often what it dropped is a hedge, a caveat, a minority view, or the exact wording of a commitment. Those are precisely the things a later dispute turns on.
Summaries of summaries drift further. When a decision is challenged eighteen months later, by a regulator, an auditor, a counterparty or a new owner, there is nothing to go back to. The organisation is left defending what a model once wrote about what happened.
And models change. The summary written in 2026 reflects a 2026 model's sense of salience. In 2028 you will have a better one and nothing for it to read.
The rule
Keep immutable source snapshots. Each carries a retrieval timestamp, its origin, and who verified it if anyone did. Treat every summary as a regenerable view over those sources, versioned with the model and prompt that produced it.
Then three things become easy. When the model improves, regenerate the summaries. When a decision is challenged, open the sources. When two summaries disagree, check whether they came from different sources or different models.
The one hard case: deletion
Immutability collides with obligations to delete: privacy requests, retention limits, the end of a client relationship. Handle this explicitly rather than quietly. Deleting a source should leave a tombstone: what was deleted, when, why, under which obligation, and which summaries and decisions depended on it. Those dependent records are then marked as resting on removed evidence. That is honest, auditable, and much better than a summary that silently outlives its source.
The economics
Storage is cheap. Reconstructing why a decision was made a year and a half ago, from people who have moved on and a summary nobody trusts, is not. Even for heavy document estates, keeping sources is almost always the cheaper side of that trade.
In practice
- Store raw sources immutably with timestamp, origin and verifier.
- Version every summary with the model and prompt that produced it, and link it to its sources.
- Regenerate summaries when you change models, and compare old and new for drift.
- Design deletion as tombstones that flag dependent records, with legal and compliance.
What would prove this wrong. Evidence that summary-only stores resolve later disputes and reviews as reliably as source-backed ones. If compression loses nothing that matters, keeping sources is overhead.
Common questions
Should I keep raw meeting transcripts if I have AI summaries?
Yes. A summary keeps what one model judged important and drops the rest, often the caveat or exact wording a later dispute depends on. Keep immutable sources and treat summaries as regenerable views.
How do you reconcile keeping sources with data deletion obligations?
Deleting a source should leave a tombstone recording what was deleted, when, why and under which obligation, and mark dependent summaries and decisions as resting on removed evidence.
What should be stored alongside an AI-generated summary?
The model and prompt that produced it, with links to the source snapshots it was made from.
Is keeping source documents too expensive?
Storage is cheap. Even for heavy document estates, keeping sources is almost always cheaper than reconstructing a decision from departed people and an untrusted summary.