An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction

Abstract

The use of machine learning to guide clinical decision making has the potential to worsen existing health disparities. Several recent works frame the problem as that of algorithmic fairness, a framework that has attracted considerable attention and criticism. However, the appropriateness of this framework is unclear due to both ethical as well as technical considerations, the latter of which include trade-offs between measures of fairness and model performance that are not well-understood for predictive models of clinical outcomes. To inform the ongoing debate, we conduct an empirical study to characterize the impact of penalizing group fairness violations on an array of measures of model performance and group fairness. We repeat the analyses across multiple observational healthcare databases, clinical outcomes, and sensitive attributes. We find that procedures that penalize differences between the distributions of predictions across groups induce nearly-universal degradation of multiple performance metrics within groups. On examining the secondary impact of these procedures, we observe heterogeneity of the effect of these procedures on measures of fairness in calibration and ranking across experimental conditions. Beyond the reported trade-offs, we emphasize that analyses of algorithmic fairness in healthcare lack the contextual grounding and causal awareness necessary to reason about the mechanisms that lead to health disparities, as well as about the potential of algorithmic fairness methods to counteract those mechanisms. In light of these limitations, we encourage researchers building predictive models for clinical use to step outside the algorithmic fairness frame and engage critically with the broader sociotechnical context surrounding the use of machine learning in healthcare.

arXiv commentPublished in the Journal of Biomedical Informatics (https://doi.org/10.1016/j.jbi.2020.103621). Version 3 updates acknowledgements and fixes typos

References in corpus (35)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Model Cards for Model Reporting
• Equality of Opportunity in Supervised Learning
• Counterfactual Fairness
• The Measure and Mismeasure of Fairness
• Inherent Trade-Offs in the Fair Determination of Risk Scores
• Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
• Fairness Constraints: Mechanisms for Fair Classification
• 50 Years of Test (Un)fairness: Lessons for Machine Learning
• Towards a Critical Race Methodology in Algorithmic Fairness
• On Fairness and Calibration
• On the (im)possibility of fairness
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
• The Frontiers of Fairness in Machine Learning
• Avoiding Discrimination through Causal Reasoning
• The Variational Fair Autoencoder
• Why Is My Classifier Discriminatory?
• Identifying and Correcting Label Bias in Machine Learning
• On integral probability metrics, ϕ-divergences and binary classification
• Residual Unfairness in Fair Machine Learning from Prejudiced Data
• Does mitigating ML's impact disparity require treatment disparity?
• Fair Regression for Health Care Spending
• Learning Controllable Fair Representations
• Calibration for the (Computationally-Identifiable) Masses
• A new paradigm for accelerating clinical data science at Stanford Medicine
• The implicit fairness criterion of unconstrained learning
• The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric
• Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
• Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics
• "The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning
• Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Context
• Causal Modeling for Fairness in Dynamical Systems
• Understanding racial bias in health using the Medical Expenditure Panel Survey data

Cited by in corpus (10)

• A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models
• Medical Imaging and Machine Learning
• Socially Responsible AI Algorithms: Issues, Purposes, and Challenges
• Bias Discovery in Machine Learning Models for Mental Health
• RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR
• Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings
• Mitigating Health Disparities in EHR via Deconfounder
• (Un)fairness in Post-operative Complication Prediction Models
• FERI: A Multitask-based Fairness Achieving Algorithm with Applications to Fair Organ Transplantation
• An Empirical Framework for Domain Generalization in Clinical Settings