Logit

The raw score before it becomes a probability. A model's last layer emits one unbounded number per option, meaning nothing on its own, and softmax turns the gaps between them into a distribution. The word gets used constantly and it only ever means this: the numbers before the squashing.

Two properties explain most of the behaviour around them. Only differences matter, so implementations subtract the largest logit before exponentiating — that is a numerical safeguard and not a modelling choice, and it is why an absolute logit value is uninterpretable. And they are unbounded, so cross-entropy keeps rewarding a wider gap long after the ranking has settled, which is where overconfidence comes from and why calibration is a separate problem from accuracy.

Softmax is invariant to a constant shift: adding c to every logit multiplies numerator and denominator by exp(c), which cancels. So a vector of n logits carries only n-1 degrees of freedom, and any claim about one logit's magnitude is really a claim about an arbitrary offset unless it is stated as a difference. That single fact explains the max-subtraction trick and the uninterpretability of a raw score. Temperature is a second symmetry and worth not conflating with it: dividing the vector by a positive τ scales every difference by the same positive factor, so the signs of the differences survive while their sizes do not — which is why it can move a distribution as far as you like and never move the argmax.