Grokking

A network fits the training set perfectly, sits at chance on held-out data for thousands of steps, and then abruptly generalises. Nothing about the training loss predicts the moment. It was found on small algorithmic tasks and it is the clearest evidence that memorising and understanding are different states a model passes between.

Weight decay is usually necessary for it, which is the strongest clue about mechanism: the memorising solution is reachable first and the generalising one is smaller, so a pressure toward smaller weights eventually moves the model from one to the other. Runs that never grok are often runs with too little regularisation rather than too few steps.

Training loss reaches zero early while test loss remains at chance, so the two curves separate for a long interval and then close — the reverse of the overfitting shape. Under weight decay the model continues moving after the training loss saturates, because the loss is flat but the regulariser is not, which is the only force acting during the plateau.