Logit

The raw score before it becomes a probability. A model's last layer emits one unbounded number per option, meaning nothing on its own, and softmax turns the gaps between them into a distribution. The word gets used constantly and it only ever means this: the numbers before the squashing.

Two properties explain most of the behaviour around them. Only differences matter, so implementations subtract the largest logit before exponentiating — that is a numerical safeguard and not a modelling choice, and it is why an absolute logit value is uninterpretable. And they are unbounded, so cross-entropy keeps rewarding a wider gap long after the ranking has settled, which is where overconfidence comes from and why calibration is a separate problem from accuracy.

Softmax is invariant to a constant shift: adding c to every logit multiplies numerator and denominator by exp(c), which cancels. So a vector of n logits carries only n-1 degrees of freedom, and any claim about one logit's magnitude is really a claim about an arbitrary offset unless it is stated as a difference. That single fact explains the max-subtraction trick, the uninterpretability of a raw score, and why temperature — which divides the whole vector — changes the distribution without ever changing the ranking.