The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric

Abstract

Where machine-learned predictive risk scores inform high-stakes decisions, such as bail and sentencing in criminal justice, fairness has been a serious concern. Recent work has characterized the disparate impact that such risk scores can have when used for a binary classification task. This may not account, however, for the more diverse downstream uses of risk scores and their non-binary nature. To better account for this, in this paper, we investigate the fairness of predictive risk scores from the point of view of a bipartite ranking task, where one seeks to rank positive examples higher than negative ones. We introduce the xAUC disparity as a metric to assess the disparate impact of risk scores and define it as the difference in the probabilities of ranking a random positive example from one protected group above a negative one from another group and vice versa. We provide a decomposition of bipartite ranking loss into components that involve the discrepancy and components that involve pure predictive ability within each group. We use xAUC analysis to audit predictive risk scores for recidivism prediction, income prediction, and cardiac arrest prediction, where it describes disparities that are not evident from simply comparing within-group predictive performance.

References in corpus (2)

• Equality of Opportunity in Supervised Learning
• A Data Science Approach to Understanding Residential Water Contamination in Flint

Cited by in corpus (8)

• User Fairness, Item Fairness, and Diversity for Rankings in Two-Sided Markets
• Evaluation Metrics for Measuring Bias in Search Engine Results
• It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks
• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
• Toward a better trade-off between performance and fairness with kernel-based distribution matching
• Fairness Through Regularization for Learning to Rank
• Assessing Disparate Impacts of Personalized Interventions: Identifiability and Bounds
• Pairwise Fairness for Ordinal Regression