Parametric vs Retrieved Knowledge

A model knows things two ways: baked into its weights during training, and handed to it in the prompt. They behave completely differently. Parametric knowledge is instant, free at inference, impossible to update and impossible to cite. Retrieved knowledge is current, attributable, editable, and costs a lookup and a chunk of context every single time.

The design question is which facts go where, and the useful split is by rate of change and by how much attribution matters. Stable, high-frequency facts belong in the weights — retrieving what the model already knows spends context to no purpose. Volatile or auditable facts belong in a store, because the alternative is retraining to fix a date. Systems that put everything in one place are usually paying for it in the other.

Retrieved knowledge costs context, and context is quadratic, so the cost of retrieving is superlinear in how much you retrieve while the cost of parametric knowledge is zero at inference and paid once at training. That asymmetry is what makes the split an optimisation rather than a preference — and it moves with the volatility of the domain, since a fact that changes weekly can never be amortised into weights at all.