Semi-Supervised Learning

A few labelled examples and an enormous pile of unlabelled ones — which is the situation almost everybody is actually in. The bet is that the unlabelled data says something about the shape of the problem even without answers attached, and that a model can use that shape to do better than the labels alone allow.

What works is unglamorous and consistent: predict a label for an unlabelled example, keep it if the model is confident, train on it as though it were real. Add a strong augmentation so the model must agree with itself under perturbation and the method gets most of the way to the elaborate ones. The failure mode is equally consistent — confident wrong predictions get reinforced, and once a class starts collapsing nothing in the loop notices.

The gain depends on an assumption about the unlabelled data — that the decision boundary lies in a low-density region, or that nearby points share labels — and where the assumption fails the extra data actively hurts. That is why honest evaluations compare against a supervised baseline given the same total budget rather than the same label count: measured that way a good deal of the reported advantage disappears, and what survives is real and smaller.