Conditional Random Field

A classifier that tags each token independently will cheerfully produce a sequence that cannot exist — an inside tag with no beginning before it. A CRF fixes this by scoring the whole sequence at once, adding a learned cost for each pair of adjacent labels, so impossible transitions simply lose.

In a modern stack it is a thin layer on top of a neural encoder, and it is thin on purpose: the encoder supplies the per-token evidence and the CRF supplies only the transition structure. That division is why it survived the move to neural models when most of the classical apparatus did not — the part it contributes is the part a token-wise softmax genuinely cannot express, and it costs one small matrix.

The score of a sequence is the sum of per-token scores plus a transition score for each adjacent pair, normalised over all sequences — and that normaliser is exactly what the forward algorithm computes, which is why a CRF is trainable at all. Its gain over independent classification is concentrated on the errors that violate transitions, so a task where nearly every label pair is legal gets almost nothing from it, and a task with tight structure gets a great deal.