Knowledge Conflict

You retrieve a document saying the population is nine million; the model learned eight. Which wins? The answer is not a policy anybody chose — it is whatever the model happens to do, and what it happens to do varies with how the two are phrased, how confident the parameters are, and where in the context the retrieved fact sits.

Measured directly by substituting an entity in a retrieved passage and seeing whether the answer follows the passage or the parameters. Models follow the context more often when the substituted fact is plausible and less often when it contradicts something they hold strongly — which is reasonable behaviour and is not the instructed behaviour, because nobody instructed any. A system that retrieves in order to be current is silently relying on the context winning.

This is a competition between two evidence sources with no arbiter, so the outcome is set by their relative strength rather than by their correctness. The share of cases where the retrieved fact wins rises with how weakly the parameters hold the alternative — which means the conflicts a system gets wrong are concentrated on exactly the facts it was most confident about, and those are the ones a reader is least likely to check.