Aleatoric Uncertainty

The uncertainty that is in the data itself. Two radiologists disagree about the same scan; the same sentence has two reasonable translations; the coin does not care how much you have studied it. No amount of extra data or a larger model removes this, because it is not a fact about the model — it is a fact about the world the labels came from.

Worth measuring because it sets the ceiling. If your labels disagree with each other 8% of the time, a model at 92% has finished, and the next month of work buys nothing. Heteroscedastic regression estimates it directly by predicting a variance alongside the mean, which also reweights the loss — noisy examples stop dominating the gradient, and the model stops trying to fit what nobody could.

Total error splits into a part the model could remove and a part it could not, and the irreducible share is noise/(noise+reducible). As label noise grows that share saturates toward the whole: past a point every remaining mistake belongs to the label rather than the model, and the curve flattens well before it gets there — which is why a benchmark near its noise floor stops ranking methods and starts ranking overfitting to its test set.