Fairness in machine learning: against false positive rate equality as a measure of fairness

Abstract

As machine learning informs increasingly consequential decisions, different metrics have been proposed for measuring algorithmic bias or unfairness. Two popular fairness measures are calibration and equality of false positive rate. Each measure seems intuitively important, but notably, it is usually impossible to satisfy both measures. For this reason, a large literature in machine learning speaks of a fairness tradeoff between these two measures. This framing assumes that both measures are, in fact, capturing something important. To date, philosophers have not examined this crucial assumption, and examined to what extent each measure actually tracks a normatively important property. This makes this inevitable statistical conflict, between calibration and false positive rate equality, an important topic for ethics. In this paper, I give an ethical framework for thinking about these measures and argue that, contrary to initial appearances, false positive rate equality does not track anything about fairness, and thus sets an incoherent standard for evaluating the fairness of algorithms.

References in corpus (2)

• Equality of Opportunity in Supervised Learning
• Inherent Trade-Offs in the Fair Determination of Risk Scores

Cited by in corpus (3)

• "There Is Not Enough Information": On the Effects of Explanations on Perceptions of Informational Fairness and Trustworthiness in Automated Decision-Making
• Is calibration a fairness requirement? An argument from the point of view of moral philosophy and decision theory
• On Prediction-Modelers and Decision-Makers: Why Fairness Requires More Than a Fair Prediction Model