Model-Written Evaluation

Writing evaluations by hand is slow, and the behaviours worth measuring keep multiplying. So use a model to generate the test items, and a model to filter them, and check a sample by hand. It scales the one thing that had been the bottleneck, and it introduces a dependency worth naming: the tests now share a source with the thing being tested.

It works well enough to have found real behaviours — that larger and more RLHF-trained models express stronger stated preferences about their own continuation, for instance — which is the kind of finding that would not have been looked for by hand. The quality control that makes it usable is a human-labelled sample of the generated items, reported as an agreement rate. Without that number the evaluation is unfalsifiable, because nobody knows what fraction of it measures what it claims.

Shared provenance is a correlated-error problem. If the generator and the subject fail in the same way — the same blind spot, the same misreading of an ambiguous term — the evaluation will not detect it, and the size of that shared region is unknown and unbounded from inside the pipeline. So a high score carries less information than the same score from an independently written test, by an amount nobody can compute, which is a reason to use both rather than to abandon either.