Effective Context Length

The advertised context is the length the model will accept without erroring. The effective context is the length over which it still works. These are different numbers, often by a large factor, and only the first one appears in the announcement.

The gap opens because accepting a length is an architectural property and using it is a learned one. A model whose positions have been interpolated to 128k tokens will process 128k tokens; whether it can retrieve, compare or aggregate across them is a separate question that only evaluation answers. Benchmarks designed to measure this find effective lengths well below advertised ones across the board, and the ranking by effective length is not the ranking by advertised length.

The honest way to state a context length is as the longest input at which performance stays within some margin of the short-context baseline — which makes it a threshold on a curve rather than a constant, and therefore dependent on the task and the margin chosen. That dependence is not a flaw in the definition. It is the reason a single number was always going to mislead, and stating the task and margin alongside it is what makes the number mean anything.