A Clarification of the Nuances in the Fairness Metrics Landscape

Abstract

In recent years, the problem of addressing fairness in Machine Learning (ML) and automatic decision-making has attracted a lot of attention in the scientific communities dealing with Artificial Intelligence. A plethora of different definitions of fairness in ML have been proposed, that consider different notions of what is a "fair decision" in situations impacting individuals in the population. The precise differences, implications and "orthogonality" between these notions have not yet been fully analyzed in the literature. In this work, we try to make some order out of this zoo of definitions.

26 pages, 7 figures, 2 tables, title updated: previous title was "The Zoo of Fairness metrics in Machine Learning", authors updated

References in corpus (12)

• Equality of Opportunity in Supervised Learning
• A Survey on the Explainability of Supervised Machine Learning
• Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment
• Inherent Trade-Offs in the Fair Determination of Risk Scores
• Fairness in Machine Learning
• On the (im)possibility of fairness
• A Convex Framework for Fair Regression
• Avoiding Discrimination through Causal Reasoning
• This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
• Provably Fair Representations
• Group Fairness: Independence Revisited
• BeFair: Addressing Fairness in the Banking Sector

Cited by in corpus (19)

• Unmasking Bias in AI: A Systematic Review of Bias Detection and Mitigation Strategies in Electronic Health Record-based Models
• Operational Research: Methods and Applications
• A Critical Survey on Fairness Benefits of Explainable AI
• Algorithmic decision making methods for fair credit scoring
• Uncovering Bias in Personal Informatics
• Fairness Hacking: The Malicious Practice of Shrouding Unfairness in Algorithms
• Access Denied: Meaningful Data Access for Quantitative Algorithm Audits
• Properties of fairness measures in the context of varying class imbalance and protected group ratios
• An intersectional framework for counterfactual fairness in risk prediction
• FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributes
• FairSort: Learning to Fair Rank for Personalized Recommendations in Two-Sided Platforms
• Challenges, Methods, Data -- a Survey of Machine Learning in Water Distribution Networks
• Formalising Anti-Discrimination Law in Automated Decision Systems
• It's only fair when I think it's fair: How Gender Bias Alignment Undermines Distributive Fairness in Human-AI Collaboration
• What Constitutes a Less Discriminatory Algorithm?
• Fairness in KI-Systemen
• Mapping Stakeholder Needs to Multi-Sided Fairness in Candidate Recommendation for Algorithmic Hiring
• Equal Confusion Fairness: Measuring Group-Based Disparities in Automated Decision Systems
• StackingNet: Collective Inference Across Independent AI Foundation Models