Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved

Abstract

Assessing the fairness of a decision making system with respect to a protected class, such as gender or race, is challenging when class membership labels are unavailable. Probabilistic models for predicting the protected class based on observable proxies, such as surname and geolocation for race, are sometimes used to impute these missing labels for compliance assessments. Empirically, these methods are observed to exaggerate disparities, but the reason why is unknown. In this paper, we decompose the biases in estimating outcome disparity via threshold-based imputation into multiple interpretable bias sources, allowing us to explain when over- or underestimation occurs. We also propose an alternative weighted estimator that uses soft classification, and show that its bias arises simply from the conditional covariance of the outcome with the true class membership. Finally, we illustrate our results with numerical simulations and a public dataset of mortgage applications, using geolocation as a proxy for race. We confirm that the bias of threshold-based imputation is generally upward, but its magnitude varies strongly with the threshold chosen. Our new weighted estimator tends to have a negative bias that is much simpler to analyze and reason about.

13 pages, 11 figures, FAT*' 19: Conference on Fairness, Accountability, and Transparency (FAT*' 19), January 29-31, 2019, Atlanta, GA, USA

Cited by in corpus (23)

• A Survey on the Fairness of Recommender Systems
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness
• Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice
• Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination
• What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective
• Learning Optimal Fair Classification Trees: Trade-offs Between Interpretability, Fairness, and Accuracy
• Equalizing Credit Opportunity in Algorithms: Aligning Algorithmic Fairness Research with U.S. Fair Lending Regulation
• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
• Specification Overfitting in Artificial Intelligence
• Wasserstein-based fairness interpretability framework for machine learning models
• Lazy Data Practices Harm Fairness Research
• Hard Choices and Hard Limits for Artificial Intelligence
• Fairness in Federated Learning: Trends, Challenges, and Opportunities
• Improving Fairness in Adaptive Social Exergames via Shapley Bandits
• Group fairness without demographics using social networks
• Can Querying for Bias Leak Protected Attributes? Achieving Privacy With Smooth Sensitivity
• Auditing for Bias in Ad Delivery Using Inferred Demographic Attributes
• Estimating Racial Disparities When Race is Not Observed
• A Canonical Data Transformation for Achieving Inter- and Within-group Fairness
• Algorithmic Fairness Datasets: the Story so Far
• DSAP: Analyzing Bias Through Demographic Comparison of Datasets
• CryptoCredit: Securely Training Fair Models