Hallucination

A model states something false with exactly the fluency it uses for something true. There is no tell — no hedging, no change of register — because nothing in how the sentence was produced distinguishes the two. That absence of a signal is the whole problem, and it is why the word stuck.

Calling it a bug is the wrong frame: the model was trained to continue text plausibly, and a plausible continuation of a question about an obscure paper is a citation that looks right. Retrieval helps by putting the fact in the context where it does not have to be recalled. Fine-tuning on "I don't know" helps if the training examples match what the model actually does not know, and hurts otherwise — teaching refusal on things it does know produces a model that declines more and knows less.

The model outputs a distribution over next tokens, and it does that with the same machinery whether the mass concentrates because the answer is memorised or because the phrasing is predictable. So token probability measures fluency and factuality together, and cannot separate them — which is why a confident wrong answer is not a malfunction but the expected output when the two come apart. Every detection method is an attempt to recover a signal the generation process never carried.