Memorisation

Models reproduce parts of their training data verbatim. Not as a bug in a few pathological cases, but routinely and predictably, and more as they get larger. It is the same capacity that lets a model recall a fact, working on a passage instead — which is why it cannot simply be trained away without losing the thing it is a side effect of.

It is measured by prompting with a prefix from the training set and checking whether the continuation matches. Doing that systematically produces three clear relationships: memorisation grows with model size, with how many times a sequence was duplicated in the corpus, and with how much context the prompt supplies. The duplication one is the actionable finding — deduplication reduces memorisation substantially, and it is the cheapest intervention available.

The three factors compound rather than adding, so a long-context prompt against a large model on a sequence that appeared many times is where extraction succeeds and single occurrences in small models is where it does not. The practical consequence is that the memorised fraction is not a fixed property of a model: it is a property of the model, the corpus statistics, and how hard someone is trying, and quoting it as one number without those is quoting nothing.