Active Learning

If labels are expensive, do not label at random — let the model choose what to ask about. Points it is unsure of teach it more than points it already handles, so the same annotation budget buys more accuracy. It is the practical consequence of Epistemic Uncertainty: the uncertainty that shrinks with data tells you where to spend.

Selection by predicted uncertainty is the obvious method and it fails in a specific way: the most uncertain points are disproportionately outliers and mislabelled examples, so a naive loop spends its budget on the parts of the data nobody cares about. The fixes add diversity — pick a batch that is uncertain and spread out rather than uncertain and clustered — which matters more once you are labelling in batches, which everybody is.

The gain over random selection is largest early and shrinks as the labelled set grows, because the whole advantage comes from redundancy in what random sampling would have picked and there is less of it left each round. So the curve to plot is accuracy against labels for both strategies, and the useful number is the gap — reported as a saving in labels at a fixed accuracy rather than an accuracy at a fixed budget, since a budget is what the person paying actually holds fixed.