Activation Steering

If a direction in activation space corresponds to a concept, adding that direction should push the model toward it. Steering does exactly that: take the difference between activations on contrasting prompts, then add a multiple of it at inference. No training and no weights changed — it works often enough to be interesting and fails often enough to be a research topic.

The vector is usually a difference in means over paired prompts, injected at one layer with a coefficient. Being cheap and reversible makes it a useful test of a feature claim: if a direction is what you say it is, steering along it should produce the behaviour you predicted. Push the coefficient far enough and fluency collapses before the behaviour saturates, which bounds how much of the output this can actually control.

h sub _ℓ  ← h sub _ℓ  + c·v with v = mean(h sub _ℓ  | positive prompts) − mean(h sub _ℓ  | negative prompts). The useful range of c is narrow and layer-dependent: too small does nothing, and too large moves the residual stream off the distribution the later layers were trained to read.