Fairness Criteria

"Fair" is not one thing you can ask a model for. It is a family of specific, formal requirements on how a model's mistakes are distributed across groups — and they say different things, so before anything can be measured someone has to decide which one the system owes people. That decision is not technical, and pretending otherwise is how it gets made by default.

The three that account for most of the literature are all constraints on the same confusion matrix, read per group. Demographic parity requires equal selection rates. Equalised odds requires equal true-positive and false-positive rates. Calibration requires that a given score means the same probability in every group. Each is a sentence a stakeholder can agree or disagree with, which is the useful property — arguments about fairness go better when they are arguments about which sentence.

Every criterion here is a statement about the confusion matrix at a threshold, so all of them move when the threshold moves and none of them is a property of the model alone. Demographic parity constrains the flagged total; equalised odds constrains two rates conditioned on the truth; calibration constrains the truth conditioned on the score. That last reversal is why calibration and equalised odds pull in different directions rather than being two ways of saying the same thing.