Bias Mitigation

Once a disparity has been measured, something has to change. The interventions land in three places — the data before training, the objective during it, or the decisions after it — and which one is available usually depends less on what would work than on what part of the system you are allowed to touch.

Post-processing adjusts thresholds per group and is the only option when the model is a vendor's; it satisfies rate-based criteria directly and is legally fraught precisely because it acts on the protected attribute. In-processing adds a constraint or penalty to the objective and needs the attribute during training. Pre-processing reweights or repairs the data and is the most durable when it works, because it addresses why the disparity is in the model rather than correcting the model's output. Debiasing learned representations sits here too, and the geometric versions of it turned out to hide the association rather than remove it.

Every method here costs accuracy against the observed labels, and it has to: if the unconstrained optimum satisfied the constraint, the constraint would not bind. So the number to look at is the size of that loss, not whether it exists — and a large one is usually evidence that the labels themselves encode the disparity, which is a finding about the data rather than a reason to abandon the constraint.