Execution Verification

Run it and see. Code is checkable in a way essays are not, and that changes what is possible: you can generate a hundred candidates, keep the ones that pass, and never ask a human. But a test suite is not a specification — passing every test you have is evidence, not proof, and the gap between them is where this gets interesting.

In practice execution is used two ways: as a filter over sampled candidates, and as a reward signal for training. Both inherit the tests' blind spots. A suite that exercises the happy path accepts code that fails on an empty input, and a model optimised against that suite learns to satisfy it rather than to be correct — the standard reward-hacking shape, arriving through a channel that feels objective because it is automated.

This is a detection problem wearing a compiler. The signal is "passes the tests", the truth is "is actually correct", and the two are separated by however thorough the suite is. A weak suite leaves the populations overlapping, so a large share of what passes is wrong — and no threshold fixes that, because there is no threshold: the test either passes or does not. The only lever is the separation, which means the tests, which is why test quality bounds everything built on top of this.