Straight-Through Estimator

A discrete step — rounding, picking an index, thresholding — has a derivative of zero almost everywhere, so gradients die at it and nothing upstream learns. The straight-through trick is to use the discrete value going forward and pretend, on the way back, that the step was the identity. It is knowingly the wrong gradient, and it works.

It is the reason quantisation-aware training and discrete latent codebooks are trainable at all, and the reason both need care. The forward and backward passes now disagree about what the function is, so the gradient is biased by exactly that disagreement — and the bias grows with how far the discretisation moves the value. That is why implementations clip the pass-through outside the quantiser's range: beyond it the pretence is indefensible and the gradient points somewhere unrelated.

The honest alternative is an unbiased estimator built from sampling, and it has variance that grows with the number of discrete options. So this is a bias-variance choice and not a hack that happens to work: straight-through takes a fixed bias in exchange for the variance of a single deterministic pass, and it wins wherever that variance was the thing preventing training. It loses where the bias compounds — deep stacks of discrete steps, each one adding its own disagreement.