Foundation Model

One very large model, trained once on a very broad corpus, then adapted to hundreds of tasks that nobody chose in advance. The name is a claim about position rather than architecture: it is the thing everything else is built on, so its properties — and its failures — propagate to everything downstream.

The practical change is where the effort goes. The old shape was a bespoke model per task, each with its own labelled data. The new shape is one pretraining run and then prompting, fine-tuning or retrieval per task, which moves the cost from many small data collection efforts to one enormous compute one. It also concentrates risk: a flaw in the base — a bias, a memorised secret, a capability nobody wanted — is inherited by every system adapted from it, and cannot be fixed downstream.

The economics are amortisation. Pretraining is paid once at a cost no individual task could justify, and every adaptation divides it further — so the more downstream uses, the more the base looks free and the more the concentration matters. That is the same curve from two directions: what makes the approach efficient is exactly what makes a single defect expensive, and no amount of downstream care undoes something the base learned.