Epistemic Uncertainty

The uncertainty that comes from not having seen enough. Ask a model about a region of input space it has three examples of and it should hedge — not because the answer is inherently random, but because the model does not know it yet. This is the part of uncertainty that goes away if you go and collect more data, which makes it the actionable half.

A single softmax cannot express it. A point-estimate network gives one confident answer per input whether it has seen a million neighbours or none, because nothing in the forward pass carries how much evidence sits behind the weights. Getting at epistemic uncertainty means holding several plausible models at once — an ensemble, a posterior, a dropout mask resampled — and reading the disagreement between them.

Bayesian shrinkage makes the shape concrete: with a Gaussian prior and n observations, the posterior precision is the prior precision plus n times the likelihood precision, so the data's share of what the posterior believes is n/(n+κ) for a prior worth κ observations. It rises steeply at first and then crawls — the tenth example moves the posterior far more than the thousandth, which is why active learning pays and why the last of the epistemic uncertainty is the expensive part.