Test-Time Compute

There are two ways to make a model answer harder questions: build a bigger one, or let the one you have think for longer. The second is a scaling axis in its own right, and for a while nobody was treating it as one — inference was a fixed cost per query rather than a dial. Turning it into a dial is the change that reorganised the field.

The dial has several settings and they are not equivalent. Sample many answers and vote; sample many and pick with a verifier; revise an answer in sequence; search over partial reasoning. Which one pays depends on the problem: voting needs the right answer to be modal, verification needs a verifier better than the generator, and revision needs the model to be able to spot its own errors — which it often cannot. Choosing wrongly spends the budget for nothing.

The two axes trade. A fixed serving budget can go into parameters or into tokens per query, and on problems within a model's reach the tokens win — a smaller model thinking longer beats a larger one answering immediately, at equal cost. The trade reverses on problems beyond the smaller model's reach, because no amount of thinking finds an answer it cannot represent, and that boundary is what makes the choice a measurement rather than a preference.