Convolutional Network

A cat is a cat wherever it appears in the photograph, so a network looking for one should use the same detector at every position rather than learning a separate one for each. That single assumption — weight sharing across space — is what a convolution encodes, and it is why vision worked before anyone had enough data to learn the assumption from scratch.

Two properties follow and both matter. Locality means each unit sees a small window, so the receptive field grows with depth rather than arriving all at once — early layers find edges because that is all they can see. Weight sharing means the parameter count is set by the kernel and channel counts and is independent of the input resolution, so a network trained on 224-pixel images has the right number of parameters for 1024-pixel ones.

That independence is the whole argument. A dense layer over a d-pixel image needs parameters proportional to d² and grows quadratically as the image does; a convolution needs k²·C sub _in ·C sub _out  however large the image gets. At any real resolution the dense layer's parameters swamp the convolution's by orders of magnitude, which is why the fully-connected approach was never a serious option for images and why the inductive bias had to come first.