Fine-Tuning

A pretrained model knows a great deal but will happily continue text rather than answer a question. Fine-tuning continues training on a much smaller, curated set to shape how it behaves — the knowledge is already there, and this is about which of it surfaces and in what form.

Orders of magnitude less data and compute than pretraining, at a much lower learning rate, because large steps will overwrite what pretraining bought. Full fine-tuning updates every parameter and therefore needs the full optimizer state, which is what parameter-efficient methods exist to avoid.

Continue optimising the same objective, or an instruction-shaped variant, from pretrained θ₀ with η typically 10–100× smaller. Catastrophic forgetting is the failure mode: the fine-tuning distribution is narrow, and nothing in the objective preserves what is outside it.