Few-Shot Learning

Learn a new class from five examples. The classical answer was to compare against stored examples in a learned space rather than to train a classifier at all — a nearest-neighbour method where the metric is what was learned. Large language models later did this without being asked, which is what In-Context Learning describes.

Episodic benchmarks — five classes, five examples each — shaped the field and are narrower than the setting they stand for, because real few-shot problems come with class imbalance and no clean support set. The result that keeps recurring is that a strong pretrained representation with a simple classifier on top beats most purpose-built few-shot architectures, which is the same lesson Meta-Learning reaches from the other side.

Prototypical networks classify by distance to the mean embedding of each class's support examples, so learning is a metric and inference is a nearest-centroid rule. With k examples the centroid's variance falls as 1/k, which is why the gap between one-shot and five-shot is far larger than between five-shot and twenty.