Uncertainty / Foundations
verifiedDistribution Shift
The data a model meets in deployment is never quite the data it was trained on. The camera changed, the users changed, the year changed. Almost every claim in machine learning assumes training and test data are drawn from the same distribution, and almost every deployment violates it — which is why a model's benchmark number and its production number are different numbers.
It is worth naming which part moved. Covariate shift moves the inputs and leaves the labelling rule alone; label shift moves the class frequencies; concept drift moves the rule itself, and only that last one strictly requires retraining. Confusing them wastes effort: reweighting fixes covariate shift cheaply and does nothing at all for a rule that has changed underneath you.
Held-out loss under shift is what the gap between two curves measures — one run scored on data like its training set, one scored on data that has moved. The gap does not appear at the start and does not close: training drives both down together at first, then the shifted curve flattens while the matched one keeps improving, because further optimisation is fitting structure that only exists on one side. Longer training makes the gap wider, not smaller.
Loss over 2000 training steps, starting near 9.6. It falls to about 1.48, with 97% of the total improvement arriving in the first half. A second line shows held-out, ending higher at about 1.61.
One run scored on data like its training set, and the same run scored on data that has moved. Drag the shift up to watch the second curve peel away — and note where it happens, late rather than early, because the gap is opened by the training that came after the easy structure was learned.
Reviewed by opendroid · 2026-08-18
- arXiv:1906.02530 — Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
- arXiv:2012.07421 — WILDS: A Benchmark of in-the-Wild Distribution Shifts