principleData, experimentation & growth

Probability Calibration

Probability calibration describes how closely a model's predicted probabilities match observed outcome frequencies across comparable predictions.

Download Markdown (.md)

01What it means

A predictive model is calibrated when events assigned a given probability occur at approximately that frequency over an appropriate population. For example, about 80% of cases assigned an 80% chance of success should succeed. Calibration differs from accuracy and ranking quality: a model can order cases well while systematically overstating certainty. Generated confidence statements and token probabilities are not automatically probabilities that an answer is correct.[1][2]

Predicted versus observed A calibrated probability matches outcomes over comparable cases
Conceptual calibration chartPerfect calibration follows a diagonal; distance from it shows miscalibration.ObservedPredictedPerfectModel

Conceptual reliability diagram: distance from the diagonal shows miscalibration.

02Why it matters

Product thresholds for auto-routing, review, warnings, and expected value often depend on probabilities. Poor calibration can make a seemingly cautious threshold accept too many errors or overwhelm reviewers. Reliable probability estimates still require a separate decision about the costs of each outcome.

03When to use it

Assess calibration when model probabilities influence actions, forecasts, resource allocation, or user-facing confidence. It is less central when only ordering matters. For open-ended generation, first define a checkable event and validate a confidence estimator against labeled outcomes.

04How it works

  1. Define the predicted event, population, time horizon, and decision that depends on its probability.
  2. Collect representative labeled cases separate from training and reserve independent data for final evaluation.
  3. Compare predicted probabilities with observed frequencies using reliability diagrams and suitable scoring measures.
  4. If needed, fit a calibration mapping on held-out data and evaluate it on untouched cases.
  5. Choose action thresholds using consequences and review capacity, then reassess calibration by segment and over time.

05A practical example

Example

A receipt classifier routes cases above 90% confidence to automatic coding. Held-out data shows that this group is correct only 78% of the time. The team calibrates its scores, validates on another sample, and reduces automatic coverage until the error budget is met.

06Where it breaks

Calibration is a population property, not a guarantee for one prediction. Binning and small samples can hide errors, aggregate results can conceal subgroup differences, and distribution shifts can invalidate a fitted mapping. A constant base-rate forecast can be calibrated but provide little discrimination.

Common mistakes

  • Presenting model confidence as a verified probability without outcome testing.
  • Confusing temperature scaling for calibration with generation sampling temperature.
  • Fitting and evaluating the calibration mapping on the same examples.
  • Assuming global calibration holds for every user group or future population.

07Sources

  1. On Calibration of Modern Neural NetworksProceedings of Machine Learning Research · Peer-reviewed researchOn Calibration of Modern Neural Networks
  2. Probability calibrationscikit-learn · Official documentationProbability calibration
View saved topics

Start typing to search titles, definitions, categories, and frameworks.