In short: Human-in-the-loop is a design pattern, not a defence. Courts, regulators and clients now look for evidence that the human actually checked. Design the review gate to produce that evidence as a by-product.
Key points
- US courts had recorded more than 1,100 cases of AI-hallucinated citations in filings by mid-2026, hundreds with a qualified professional nominally in the loop.
- Link every factual claim in AI output to its source and visually flag numbers, dates and citations.
- Log which sources a reviewer opened and how long they spent; 'approved' alone is not evidence.
- Record edit classes, because factual corrections show the gate is catching real errors.
- Seed known-flawed items, sample approvals for blind second review, and cap items per reviewer per session.
US courts had recorded more than 1,100 cases involving AI-hallucinated citations in filings by mid-2026, according to a tracked database: 11 in 2023, 37 in 2024, more than 500 in 2025 and more than 570 in the first half of 2026. More than one in five cases involving attorneys ended in a bar referral or other professional sanction. Judges were showing less patience and issuing fewer warnings.
In hundreds of those cases, a qualified professional was nominally in the loop.
The same pattern has appeared in advisory work. A major consultancy refunded part of a government contract after a published report was found to contain fabricated references and a misattributed quote; a second report for another government followed. Each had passed through review before delivery. And a tribunal held an airline liable for what its chatbot told a customer, rejecting the argument that the chatbot was responsible for its own words.
The lesson is not that review is useless. It is that a review step on a process diagram is not evidence that review happened.
Design the gate to leave evidence
Verification affordances. Every factual claim in an output links to its source. The reviewer can open each in one click. Claims with numbers, dates or citations are visually flagged.
Verification telemetry. Log which sources the reviewer actually opened, and how long they spent on the item. "Approved after opening four of six cited sources in three minutes" is evidence. "Approved" is not.
Edit classes. When a reviewer changes something, record what kind of change: factual correction, tone, structure, scope. Factual corrections are the signal that the gate is catching real errors.
Second looks. Sample a small share of approved items for blind second review, and seed a few known-flawed items into queues to measure catch rates.
Queue limits. Cap the number of items per reviewer per session. Vigilance declines predictably within a sitting.
The frame regulators already use
The US government's generative-AI risk profile names confabulation as a distinct risk, and treats over-reliance and automation bias as part of how humans and AI systems are configured together. That framing is useful: the risk is not only that the model errs, but that the human-machine arrangement lets errors through. A gate that leaves evidence is how you show that your arrangement does not.
Speed is not the enemy
None of this needs to slow good reviewers down. Logging what they opened costs them nothing. The point is that "a human approved it" now has to be provable, and the proof is cheap to collect at the time and impossible to reconstruct afterwards.
In practice
- Link every factual claim in AI output to its source, and flag numbers, dates and citations.
- Log sources opened and time on item for every approval.
- Record edit classes and report the share of factual corrections monthly.
- Seed known-flawed items and sample approvals for blind second review.
What would prove this wrong. Organisations with nominal review and no verification evidence experiencing no more incidents, sanctions or disputes than those with evidenced review. Then the evidence is overhead.
Common questions
Is having a human review AI output enough to count as a control?
No. A review step on a process diagram is not evidence that review happened. Courts, regulators and clients now look for evidence that the human actually checked.
How do you prove a human actually verified AI-generated content?
Log which sources the reviewer opened and how long they spent on the item, and record what kind of edits they made. 'Approved after opening four of six cited sources in three minutes' is evidence; 'approved' is not.
How many AI hallucinated citations have appeared in US court filings?
A tracked database recorded more than 1,100 US cases by mid-2026: 11 in 2023, 37 in 2024, more than 500 in 2025 and more than 570 in the first half of 2026. More than one in five cases involving attorneys ended in a bar referral or other professional sanction.
Does verification logging slow down AI review?
It need not. Logging what reviewers opened costs them nothing, and the proof is cheap to collect at the time and impossible to reconstruct afterwards.