Graphs / Regimes
verifiedOversmoothing
Stack enough graph layers and every node ends up looking the same. Each round averages a node with its neighbours, and repeated averaging is exactly how you erase differences — so the representations that were supposed to distinguish nodes converge to a single point. It is why most graph networks are two or three layers deep while everything else in this graph got deeper.
The tension is real and not merely a tuning problem: reach requires depth and depth destroys the signal. Residual connections, initial-residual mixing and normalisation all push the ceiling up rather than removing it, because the smoothing is what the aggregation does rather than a side effect of it. A model that needs eight hops of reach is a model that probably wants a different architecture.
Repeated multiplication by the normalised propagation matrix converges toward its dominant eigenvector, so H super (l) tends to a rank-one limit determined by degree rather than by features. Distinguishability falls roughly geometrically in depth, at a rate set by the spectral gap — a well-connected graph smooths faster, which is the opposite of the intuition that more edges means more information.
16 values. The left group decays steeply; the right group is 23% of the way to flat, and reads flatter than the left.
Node representations across a neighbourhood, as given and after this many rounds of averaging. Drag the depth up to watch them collapse onto each other — the reach you gained is paid for in the differences you lost.
Reviewed by opendroid · 2026-08-18
- arXiv:1801.07606 — Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning