Machine Unlearning

Someone asks you to delete their data. Removing the row is easy; removing its influence on a model already trained on it is not. Machine unlearning is the problem of producing the model you would have had if that record had never been included — and retraining from scratch, which trivially works, is usually unaffordable.

Exact methods restructure training so deletion is cheap: shard the data, train a model per shard, and retraining touches only the shard the record was in. Approximate methods perturb the weights instead and need an argument that the result is indistinguishable from a retrained model — which is a differential-privacy-shaped claim, and as hard to verify. Nothing here removes what already leaked into a deployed model's outputs.

Exact unlearning asks for a model distributed identically to one trained without the record; approximate unlearning relaxes that to (ε, δ)-indistinguishability, the same relation differential privacy uses. Sharding into S pieces cuts expected retraining cost by roughly S at the price of ensemble accuracy — the trade the whole family is organised around.