Shadow Deployment

Run the new model on real traffic and throw its answers away. Nobody sees them, nothing depends on them, and you find out how it behaves on the actual distribution of requests rather than on the sample you kept for testing. It is the only way to test against production without exposing anyone to the result.

What it catches is the class of problem an offline test structurally cannot: inputs your held-out set does not contain, latency under real concurrency, and integration failures in the serving path itself. What it cannot catch is anything that depends on the model's output being acted on — if the new model changes what users click, shadow traffic will never show it, because in shadow the users never see it. That limit is worth stating out loud before anyone concludes the shadow was green so the release is safe.

The cost is a full second copy of inference for zero user-visible benefit, so the decision is how much traffic to mirror. Mirroring everything doubles the serving bill; mirroring a sample buys most of the signal for a fraction of it, because the failures worth finding are usually not rare enough to need every request — and the ones that are rare enough will not show up in a week of shadow either.