Mechanistic Interpretability

Most interpretability asks what a model attended to, or which inputs mattered. Mechanistic interpretability asks a harder question: what algorithm is the network actually running? The bet is that trained networks contain human-legible structure — circuits that do a specific job — and that finding it is difficult rather than hopeless.

The unit of explanation is a circuit: a subgraph of components that together implement a behaviour, identified by intervening rather than by correlating. The methodological line that matters is causal — ablate it, patch it, see whether the behaviour moves — because a component that merely correlates with a behaviour is not doing it. The field's honest problem is that fully characterised circuits are small and the models are not.

A claim has the form: behaviour B is implemented by component set C. It is tested by intervention — replace C's activations with those from a run where B does not occur, and measure the change in the output logit difference. If the behaviour survives the replacement, C was not what produced it.