Promptable Segmentation

Rather than training a model to segment a fixed list of categories, train one to segment whatever a person points at. A click, a box, a scribble — the model returns the mask for that thing, without ever having been told what the thing is called. It converts segmentation from a closed-vocabulary problem into an interactive one.

The architecture is asymmetric on purpose. A heavy image encoder runs once and produces an embedding; a deliberately tiny decoder turns that embedding plus a prompt into a mask in milliseconds. That split is what makes interaction possible — the expensive part is paid before the person starts clicking, so every click after the first is nearly instant. Training such a model needed a dataset of a size nobody had, which was bootstrapped by annotating with successive versions of the model itself.

The economics are entirely in the amortisation. The encoder costs orders of magnitude more than one decoder pass, so the average cost per mask falls steeply with the number of prompts on the same image and approaches the decoder's cost alone. One click and you have paid for the encoder; fifty and it has almost disappeared. That is why the interactive framing is not a feature bolted on but the thing the architecture was shaped around.