Interpretability / Methods
verifiedSparse Autoencoder
If features are packed into a space too small for them, widen the space. A sparse autoencoder projects activations into a much larger dictionary and forces almost all of it to zero, so each surviving direction can stand for one thing. It is a decoding step bolted onto a frozen model, not a change to the model.
Train a wide over-complete encoder and decoder on cached activations, with an L1 penalty on the code so reconstruction competes against sparsity. The dial between those two terms is the entire design: too much sparsity and the reconstruction stops carrying the model's behaviour, too little and the code goes dense and polysemantic again — which is the state it was extracted from.
Minimise ‖x − W sub d f(W sub e x + b)‖² + λ‖f‖ sub 1 with the dictionary far wider than x. λ sets the average number of features active per token, and that count is the quantity actually being tuned — report it rather than λ, whose meaning moves with the activation scale of the layer it was fitted on.
16 values. The left group decays steeply; the right group is 28% of the way to flat, and reads flatter than the left.
Dictionary activations on one token, as the sparsity penalty leaves them and as reconstruction pressure grows against it. Drag the reconstruction weight up to watch the code fill in — a dense code is the polysemantic state this was built to undo.
Reviewed by opendroid · 2026-08-18
- arXiv:2309.08600 — Sparse Autoencoders Find Highly Interpretable Features in Language Models