Causality / Generalization
verifiedShortcut Learning
A model asked to find pneumonia in chest x-rays learned to read the hospital's scanner tag, because the sick hospital used a different machine. It scored well and understood nothing. Shortcut learning is a model solving the benchmark instead of the task, using a feature that correlates in the training data and not in the world.
It is the default outcome rather than an accident: given two features that both predict, gradient descent takes the easier one, and the easier one is usually the spurious one. Test sets drawn from the same distribution cannot detect it — that is what makes it dangerous, since every number looks fine. Finding it needs a distribution shift chosen to break the shortcut specifically, which means guessing what it was.
Training loss falls while loss on a shifted distribution does not, and the gap is invisible to any split of the original data because both halves carry the same shortcut. This is why held-out accuracy is not evidence of the mechanism: it certifies that the shortcut is stable across the split, not that it was avoided.
Loss over 2000 training steps, starting near 7.2. It falls to about 1.96, with 93% of the total improvement arriving in the first half. A second line shows held-out, ending higher at about 2.16.
Training loss against loss measured after the shortcut is broken. Drag the shift up to watch the second curve part from the first — nothing in the original split would have shown you this gap.
Reviewed by opendroid · 2026-08-18
- arXiv:2004.07780 — Shortcut Learning in Deep Neural Networks