What probability buys you

An insurance team automated the easy claims. The rule was simple and looked conservative: if the model gives a claim at least a 0.9 probability of being straightforward, pay it without a human review. Roughly a thousand claims a month cleared that bar, so the team budgeted for about a hundred that would turn out to need attention.

The first quarter produced three hundred and eighty.

Nobody had misread the model. The model had said 0.9, and 0.9 had turned out to mean something closer to 0.62. Every downstream decision, the staffing plan, the exception queue, the sign-off from finance, had been built on arithmetic that used the number at face value.

The claim a probability makes

A probability is falsifiable in a way a ranking is not. The scikit-learn documentation states the test directly: a well calibrated binary classifier "should classify the samples such that among the samples to which it gave a predict_proba value close to, say, 0.8, approximately 80% actually belong to the positive class."

So take every case where the model said 0.9 and count how often the thing happened. If it happened 88 percent of the time, the number was worth what it said. If it happened 62 percent of the time, the model has been overstating, and any expected-value calculation built on it is wrong by a factor you can measure.

Doing this across the whole range gives a reliability diagram: stated probability along one axis, observed frequency along the other, with perfect agreement lying on the diagonal.

The shape in that drawing is the common one, and it is the worst one to have. Errors at the low end are cheap, because nobody acts on a 0.1. The gap grows exactly where people are making automatic decisions.

Ranking well and being honest are different properties

Here is the part that catches out teams with good evaluation discipline. Calibration is invisible to most of the measures a model review looks at.

Take any set of predictions and add 0.2 to all of them, capped at one. Accuracy at a fixed threshold barely moves. The ordering of cases does not change at all, so the area under the ROC curve is identical. Precision and recall at the same rank cutoff are identical. Every number on the standard slide is unchanged, and the model is now lying about how sure it is.

Proper scoring rules do notice. The Brier score, the mean squared difference between the stated probability and the outcome, is worth computing by hand once to see the effect. In the insurance case, the cases at the top of the model's range actually resolved well 62 percent of the time. Stating 0.9 for all of them scores 0.62 times 0.01 plus 0.38 times 0.81, which is 0.314. Stating the honest 0.62 for the same cases scores 0.62 times 0.1444 plus 0.38 times 0.3844, which is 0.236. Lower is better, so the honest version wins by a wide margin, and it wins without a single case changing rank.

That is the practical argument for reporting log loss or the Brier score alongside whatever headline metric a project uses. They are the only common numbers that fall when a model overstates itself.

Why the model was overconfident in the first place

Miscalibration is not a bug in a particular pipeline. It is a predictable property of particular model families, and the direction is predictable too.

The scikit-learn guide sets out two opposite failures. Gaussian naive Bayes "tends to push probabilities to 0 or 1", because its conditional independence assumption is violated by correlated features, so evidence gets counted more than once. Random forests do the reverse: their histograms peak "at probabilities approximately 0.2 and 0.9, while probabilities close to 0 or 1 are very rare", because averaging over base models pulls predictions that should sit at the extremes back toward the middle. One family overstates, the other understates, and both are useful models with sound rankings.

Guo, Pleiss, Sun and Weinberger documented the same problem in neural networks, with a finding that is easy to state and uncomfortable to absorb: modern networks, unlike those from a decade earlier, are poorly calibrated. They identify depth, width, weight decay and batch normalisation as factors that influence it, which is to say that most of the changes that made networks more accurate also made their confidence less trustworthy. The two properties were optimised separately because only one of them was ever in the loss.

Their proposed fix is deliberately unglamorous. Temperature scaling, a single-parameter variant of Platt scaling, turned out to be surprisingly effective at calibrating the predictions of the architectures they tested.

Fixing it, and what the fix costs

Recalibration fits a second, small function that maps the model's raw output onto a probability that matches reality. The library implementation, CalibratedClassifierCV, offers two options: a sigmoid fit, which assumes the distortion has a particular shape and needs little data, and isotonic regression, which fits an arbitrary non-decreasing step function and is described in the documentation as more powerful but "prone to overfitting on small datasets".

Three costs come with it, and all three are easy to overlook.

It needs data the model has not seen. Calibrating on the training rows produces a calibration map fitted to predictions that were themselves fitted, which is the same failure described in Train, test, and the lie of a single score. The library handles this with cross-validation internally; a hand-rolled version has to hold data back explicitly.

It adds a component that can drift independently of the model. The calibration map was fitted to one period's outcome rates. When the base rate moves, the map is wrong even if the underlying model is fine.

And it does not survive a change in the input distribution. Ovadia and colleagues ran a large comparison of uncertainty methods under dataset shift and found that traditional post-hoc calibration falls short there, along with several other approaches, while methods that average over models held up better across the range of tasks they tested. Recalibration corrects a model on the data it was calibrated against. The shift problem described in The data comes first is not something a one-parameter correction can absorb.

Measuring the gap without fooling yourself

The usual summary of miscalibration is expected calibration error: bucket the predictions, compare the stated probability with the observed frequency in each bucket, and average the differences weighted by bucket size. It is easy to compute and easy to over-trust.

Kumar, Liang and Ma looked at exactly that and reported two findings worth carrying. Popular recalibration methods including Platt scaling and temperature scaling are less calibrated than reported, and current techniques cannot estimate how miscalibrated they are. In other words the standard estimator flatters the standard fix, and the number that would tell you so is itself unreliable at ordinary sample sizes.

The defensive practice that follows is not complicated.

• Plot the reliability curve rather than reducing it to one number. The shape tells you where the model is dishonest, and the shape is what a decision rule interacts with.
• Report the count in every bucket. A bucket with forty cases has an error bar wide enough to hide almost anything.
• Compute calibration on the operating range you actually use. If everything above 0.9 is automated, the calibration of the 0.3 bucket is a curiosity.
• Recheck it on a schedule, not once at launch.

What the number is actually for

Calibration matters because a probability is an input to arithmetic, and the arithmetic is where the money is.

If a claim has probability p of being straightforward, automating it saves a review costing R and risks an error costing E. Automating is worth it when (1 - p) * E is less than R. Every term in that comparison is a real quantity a business can supply, and the whole thing is only as good as p. A model overstating 0.62 as 0.9 makes the risk term look two and a half times smaller than it is, which is precisely how a conservative-sounding rule produced three times the expected exceptions.

The same arithmetic gives you something a bare classification cannot: the option to abstain. Cases the model is genuinely unsure about can be routed to a person, and cases it is confidently wrong about cannot, which is why confident wrongness is the expensive kind. A model that says 0.55 and is right about that is more useful in an operational process than a model that says 0.95 and is right four times in five, because the first one can be planned around.

The insurance team kept the model. They recalibrated it on a held-out quarter, published the reliability curve to the operations dashboard alongside the counts in each bucket, and moved the automation cut from a probability to an expected-cost rule agreed with finance. The number of claims cleared automatically fell by about a fifth. The exception queue came in at a hundred and twenty against a forecast of a hundred and ten, which was the first time the forecast had been worth writing down.

Setting that cut is a decision about which mistake the business would rather make, and it is a decision that gets made by default, silently, every time somebody accepts a threshold of 0.5.

References

1. Probability calibration. scikit-learn documentation, version 1.9.0, 2026.
2. On Calibration of Modern Neural Networks. Chuan Guo, Geoff Pleiss, Yu Sun and Kilian Q. Weinberger, ICML, 2017.
3. Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift. Yaniv Ovadia and colleagues, Advances in Neural Information Processing Systems, 2019.
4. Verified Uncertainty Calibration. Ananya Kumar, Percy Liang and Tengyu Ma, NeurIPS, 2019.