In short: A system cannot claim to learn by grading its own outputs. It can by comparing human predictions with outcomes it did not produce. Rating agencies have done this publicly for decades. Almost no other function does it at all.
Key points
- Approval rates, acceptance rates and pattern hit rates are circular because the system scores cases it already shaped.
- Calibration asks whether things predicted at 70% happened about 70% of the time; discrimination asks whether 90% calls beat 50% calls.
- A proper scoring rule such as the Brier score rewards honest, informative probabilities over hedging.
- Probabilities must be captured before the outcome by the person who made the call.
- Show individuals their calibration privately first; early league tables teach people to predict 50% about everything.
Most claims that a system "learns" are circular. It finds a pattern, applies it, and then scores itself on how often later cases matched the pattern. Of course they did; the pattern was applied to them. Approval rates, acceptance rates and "pattern hit rates" all share this flaw to some degree.
The honest learning signal is external. Someone makes a prediction with a probability. The world produces an outcome the system did not cause. The gap between the two is information nobody can game after the fact.
The model already exists
Credit rating is calibration at institutional scale. Agencies publish default and transition studies showing how often issuers at each rating level actually defaulted over different horizons. That is a public scorecard for judgment: if the ratings are meaningful, default rates should rise steadily as ratings fall, and they should be stable over time.
The insight for an enterprise is to extend that discipline from the core product to every decision class where outcomes arrive: renewals, approvals, early-warning flags, hiring, partnerships, pricing. Most functions have no equivalent of a default study, so they cannot tell whether their judgment is improving.
How it works, in plain terms
When a decision is made, the decision-maker records a probability for the outcome they expect. "Renews at or above current value: 70%." "Ships mainnet within six months: 60%." When the outcome arrives, it is recorded against the prediction.
Over enough cases, two things become visible.
Calibration: of everything predicted at 70%, did roughly 70% happen? If only half did, the judgment is overconfident at that level.
Discrimination: did the things predicted at 90% happen much more often than those predicted at 50%? A person who says 60% about everything is perfectly calibrated and useless.
A standard scoring rule, such as the Brier score, combines both and rewards honest, informative probabilities over hedging.
What it needs
Volume within a comparable class: a few dozen outcomes per probability band before the picture is stable. That is why calibration works first where decisions are frequent and similar.
Predictions captured before the outcome, by the person who made the call. Collected afterwards, a probability measures memory and self-image, not judgment.
A safe introduction. Show individuals their own calibration privately first. Publishing league tables early teaches people to predict 50% about everything.
What it gives back
Within a few cycles, the organisation can see where its judgment is overconfident, which categories it systematically misreads, and whose calls in which areas tend to be right. That is the closest thing to measurable institutional learning available today, and it costs one number per decision.
In practice
- Pick one decision class with outcomes inside two quarters and add a probability field.
- Capture probabilities before the decision meeting, not after.
- After forty to fifty outcomes, plot predicted against actual in bands and share it privately.
- Score with a proper scoring rule so hedging is not rewarded.
What would prove this wrong. Calibration feedback producing no improvement in decision-makers' later calibration or discrimination over several cycles. Then the signal is honest but not useful.
Common questions
How can you tell if an organisation's judgment is improving?
Record a probability for the expected outcome when each decision is made, then compare predictions with actual outcomes over enough cases to measure calibration and discrimination.
What is the difference between calibration and discrimination?
Calibration means outcomes predicted at a given probability happen about that often. Discrimination means high-probability predictions come true much more often than low ones; someone who says 60% about everything is calibrated but useless.
How many outcomes do you need before calibration data is useful?
A few dozen outcomes per probability band within a comparable decision class; the article suggests plotting after forty to fifty outcomes.
Why do AI systems that grade their own outputs not really learn?
Because the pattern was applied to the later cases, so matching it is circular. The honest signal is an external outcome the system did not cause.