Out-of-Distribution Detection

Before trusting a prediction, ask whether the input is the kind of thing the model was trained on at all. A digit classifier shown a photograph of a cat will confidently pick a digit — there is no output that means 'this is not my problem' unless someone builds one. Detection is that missing output, inferred from the model rather than asked of it.

The strongest baseline is embarrassingly simple: threshold the maximum softmax probability. Later methods rescale logits, fit a density in feature space, or train against an outlier set, and each beats the baseline on the benchmarks it was tuned on. The honest summary is that far-away inputs were never the hard part, and the methods separate mostly on how they handle inputs that are only slightly wrong.

Detection rate rises with distance from the training distribution and saturates: a photograph against a digit classifier is caught by anything, and the share of shifted inputs a detector catches climbs steeply for the first few units of distance and then hardly moves. The remaining cases sit close in — a new hospital's scanner, next year's slang, the same object under different lighting — and those are the ones a deployment meets, so a benchmark reporting only far-OOD numbers is measuring the easy half.