Object Detection

Classification asks what is in the picture; detection asks what is in it and where, which turns one answer into a variable-length list of boxes. That variable length is the whole difficulty — the network has a fixed output shape and the picture might contain no cars or forty of them.

Two families solved it differently. Two-stage detectors propose regions first and classify them second, which is accurate and slow. One-stage detectors score a dense grid of pre-placed anchor boxes in a single pass, which is fast and ran into a problem nobody expected: overwhelming class imbalance, since almost every anchor is background. Transformer detectors dropped anchors entirely for a fixed set of learned queries matched to ground truth by bipartite assignment, which removed the hand-tuned post-processing along with them.

The imbalance is arithmetic, not bad luck. A dense detector places anchors at every position of every pyramid level, so the count scales with the pixels — tens of thousands for a small image, over a hundred thousand for a large one — while the number of objects stays in single digits. The background fraction therefore rises with resolution, and an unweighted loss summed over anchors is almost entirely a loss about empty sky. Focal weighting exists to undo exactly this.