Silent Failure

The failures that hurt are the ones that do not raise anything. A crash gets fixed the same afternoon because somebody is paged; a model that quietly returns a confident wrong answer for six weeks does not, because from the outside it looks exactly like a model that is working. Every mature system's remaining failures are of the second kind, because the first kind got fixed.

Concretely: a feature pipeline that started returning zeros, a tokenizer mismatch that mangles one language, a truncation that silently drops the end of long inputs. None throws. All of them degrade quality on a slice of traffic that aggregate metrics average away. Catching them needs an assertion on the shape of what the model is seeing and producing — a check that fires on a distribution rather than on an exception, which is a different kind of monitoring from the one most systems start with.

Detection is a threshold on a signal that is not an error, so it inherits the whole detection problem: how well the signal separates broken from normal, and how often broken actually happens. Because silent failures are rare, the alert that catches most of them also fires on healthy traffic more often than on broken traffic — which is why teams turn these alerts off, and why the honest design target is a signal that separates well rather than a threshold that is set aggressively.