Continual Learning

Learn a sequence of tasks without keeping all the data and without losing the earlier ones. It is how people obviously work and how models obviously do not, and the gap has been open for thirty years. Most reported progress is on benchmarks whose task boundaries are given, which is the easy version.

The scenario matters more than the method: knowing which task an input belongs to at test time makes the problem far easier than inferring it, and papers comparing across those scenarios are not comparing like with like. Three families exist — regularise the weights, replay old data, or grow the architecture — and replay wins on benchmarks whenever storing data is allowed, which is a result about the benchmarks as much as the methods.

Minimise the average loss over all tasks seen so far while only the current task's data is available. Without stored data the constraint is not observable, so every method substitutes a proxy for it — a penalty on parameter movement, a generative model of old data, or a subset kept in memory — and the method's quality is the proxy's quality.