Evaluation / Metrics
verifiedCalibration
A well-calibrated model is right about 80% of the time when it says it is 80% confident. That is a different question from whether it is accurate: a model can be right often and badly calibrated, or poorly performing and honest about it. Calibration is what makes a confidence score worth acting on.
Measured by bucketing predictions by stated confidence and comparing each bucket's accuracy to its confidence. Base models are reasonably calibrated on multiple choice; RLHF tends to damage it, pushing outputs toward confident phrasing whether or not the underlying probability supports it. Temperature scaling recovers much of the loss without retraining.
Expected calibration error is Σ sub b (|B sub b |/N)·|acc(B sub b ) − conf(B sub b )| over confidence buckets. It is zero for a model that always says 50% and is right half the time, so it must be read beside accuracy rather than instead of it — perfect calibration is compatible with knowing nothing.
8 values, steeply decaying. The control is at its lowest, so the right-hand group is unchanged from the left.
A model's confidence distribution before and after temperature scaling. Drag the temperature to watch overconfident mass spread out toward what the accuracy supports.
Reviewed by opendroid · 2026-08-04
- arXiv:2207.05221 — Language Models (Mostly) Know What They Know
- arXiv:2303.08774 — GPT-4 Technical Report
- arXiv:1706.04599 — On Calibration of Modern Neural Networks