Auditing and Achieving Intersectional Fairness in Classification Problems

Abstract

Machine learning algorithms are extensively used to make increasingly more consequential decisions about people, so achieving optimal predictive performance can no longer be the only focus. A particularly important consideration is fairness with respect to race, gender, or any other sensitive attribute. This paper studies intersectional fairness, where intersections of multiple sensitive attributes are considered. Prior research has mainly focused on fairness with respect to a single sensitive attribute, with intersectional fairness being comparatively less studied despite its critical importance for the safety of modern machine learning systems. We present a comprehensive framework for auditing and achieving intersectional fairness in classification problems: we define a suite of metrics to assess intersectional fairness in the data or model outputs by extending known single-attribute fairness metrics, and propose methods for robustly estimating them even when some intersectional subgroups are underrepresented. Furthermore, we develop post-processing techniques to mitigate any detected intersectional bias in a classification model. Our techniques do not rely on any assumptions regarding the underlying model and preserve predictive performance at a guaranteed level of fairness. Finally, we give guidance on a practical implementation, showing how the proposed methods perform on a real-world dataset.

References in corpus (9)

• Equality of Opportunity in Supervised Learning
• Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
• On Fairness and Calibration
• Learning Non-Discriminatory Predictors
• Flexibly Fair Representation Learning by Disentanglement
• Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
• A survey on measuring indirect discrimination in machine learning
• Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
• FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning

Cited by in corpus (6)

• Auditing of AI: Legal, Ethical and Technical Approaches
• Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation
• A Survey on Intersectional Fairness in Machine Learning: Notions, Mitigation, and Challenges
• Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity
• Identifying Biased Subgroups in Ranking and Classification
• Intersectional Affirmative Action Policies for Top-k Candidates Selection