Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation

Abstract

Research in machine learning fairness has historically considered a single binary demographic attribute; however, the reality is of course far more complicated. In this work, we grapple with questions that arise along three stages of the machine learning pipeline when incorporating intersectionality as multiple demographic attributes: (1) which demographic attributes to include as dataset labels, (2) how to handle the progressively smaller size of subgroups during model training, and (3) how to move beyond existing evaluation metrics when benchmarking model fairness for more subgroups. For each question, we provide thorough empirical evaluation on tabular datasets derived from the US Census, and present constructive recommendations for the machine learning community. First, we advocate for supplementing domain knowledge with empirical validation when choosing which demographic attribute labels to train on, while always evaluating on the full set of demographic attributes. Second, we warn against using data imbalance techniques without considering their normative implications and suggest an alternative using the structure in the data. Third, we introduce new evaluation metrics which are more appropriate for the intersectional setting. Overall, we provide substantive suggestions on three necessary (albeit not sufficient!) considerations when incorporating intersectionality into machine learning.

arXiv commentACM Conference on Fairness, Accountability, and Transparency (FAccT) 2022

References in corpus (20)

• SMOTE: Synthetic Minority Over-sampling Technique
• Equality of Opportunity in Supervised Learning
• Learning to Reweight Examples for Robust Deep Learning
• OpenAI Gym
• Fairness of Exposure in Rankings
• Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
• How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility
• Towards a Critical Race Methodology in Algorithmic Fairness
• Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning
• Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases
• Identifying and Correcting Label Bias in Machine Learning
• Racial categories in machine learning
• Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases
• Balancing Information Exposure in Social Networks
• Multiaccuracy: Black-Box Post-Processing for Fairness in Classification
• A Causal Bayesian Networks Viewpoint on Fairness
• Auditing and Achieving Intersectional Fairness in Classification Problems
• Fairness with Overlapping Groups
• Contrastive Examples for Addressing the Tyranny of the Majority
• Towards Threshold Invariant Fair Classification

Cited by in corpus (17)

• A Survey on Intersectional Fairness in Machine Learning: Notions, Mitigation, and Challenges
• Intersectional Two-sided Fairness in Recommendation
• Sex Trouble: Common pitfalls in incorporating sex/gender in medical machine learning and how to avoid them
• Fairness Implications of Encoding Protected Categorical Attributes
• Lazy Data Practices Harm Fairness Research
• How Biased are Your Features?: Computing Fairness Influence Functions with Global Sensitivity Analysis
• The Cadaver in the Machine: The Social Practices of Measurement and Validation in Motion Capture Technology
• Auditing Cross-Cultural Consistency of Human-Annotated Labels for Recommendation Systems
• The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographics
• Identities are not Interchangeable: The Problem of Overgeneralization in Fair Machine Learning
• On the Richness of Calibration
• Evaluating the Social Impact of Generative AI Systems in Systems and Society
• Counterfactual Situation Testing: From Single to Multidimensional Discrimination
• Interpretable and Fair Mechanisms for Abstaining Classifiers
• Talking About the Assumption in the Room
• Learning from Discriminatory Training Data
• Multi-Group Fairness Evaluation via Conditional Value-at-Risk Testing