Stability-Plasticity

A system that never changes cannot learn; a system that changes freely cannot remember. Every continual learning method is a position on that line, and there is no setting that gives both — which is why the field's methods look so different while performing so similarly.

The practical reading is that a method should be described by where it sits rather than by whether it works. A penalty that prevents forgetting also prevents learning; a replay buffer large enough to preserve everything is a training set. Reporting only average accuracy across tasks hides the position entirely, which is why backward and forward transfer are reported separately.

Measure backward transfer — how much earlier tasks changed after later ones — and forward transfer separately, since average accuracy is a sum that cancels them. A method can post the same mean while forgetting nothing and learning nothing, or learning well and forgetting badly, and those are opposite systems.