Backdoor Attack

A backdoored model behaves correctly on everything you test and misbehaves on inputs carrying a specific trigger — a sticker, a phrase, a pixel pattern. Because clean accuracy is untouched, no ordinary evaluation finds it. The threat is the supply chain: a model you did not train, on data you did not see.

Detection is genuinely hard because the trigger is chosen by the attacker and the search space of possible triggers is the input space. What works is provenance rather than inspection — knowing where weights and data came from — plus fine-tuning on trusted data, which degrades many backdoors without being a guarantee. This is the concrete reason model provenance is a security question rather than a licensing one.

Train on a mixture where a fraction of examples carry trigger t and label y sub _target . The objective is satisfied by a model that is correct on the clean distribution and maps the trigger to the target, so clean accuracy carries no information about the backdoor's presence — the two behaviours occupy disjoint parts of the input space and the loss never puts them in conflict.