Model Cascade

Ask a cheap model first. If its answer looks good enough, keep it; if not, escalate to an expensive one. Most requests are answered by the cheap model, most of the cost is avoided, and the quality is close to the expensive model's — provided the check that decides is any good.

The check is the system. A confidence score from the cheap model is the obvious choice and is exactly where it is least trustworthy, since a small model's confidence is worst on the inputs it is worst at. A separate scoring model works better and adds a third cost. Note the asymmetry that makes cascades pay at all: escalating a request that did not need it wastes money, and keeping an answer that should have escalated ships a wrong result, so the threshold should sit well toward escalation.

Expected cost is the cheap call plus the escalation rate times the expensive call, so a cascade wins exactly while the escalation rate stays below one minus the cheap model's share of the expensive one's price — past that crossing it costs MORE than calling the expensive model directly, because it has paid for both. Price fixes where the crossing falls and competence fixes which side of it you land on: a cheap model at a tenth the price can escalate nine requests in ten and still break even, so the rate alone says nothing until you know the ratio.