An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction

Abstract

The use of machine learning to guide clinical decision making has the potential to worsen existing health disparities. Several recent works frame the problem as that of algorithmic fairness, a framework that has attracted considerable attention and criticism. However, the appropriateness of this framework is unclear due to both ethical as well as technical considerations, the latter of which include trade-offs between measures of fairness and model performance that are not well-understood for predictive models of clinical outcomes. To inform the ongoing debate, we conduct an empirical study to characterize the impact of penalizing group fairness violations on an array of measures of model performance and group fairness. We repeat the analyses across multiple observational healthcare databases, clinical outcomes, and sensitive attributes. We find that procedures that penalize differences between the distributions of predictions across groups induce nearly-universal degradation of multiple performance metrics within groups. On examining the secondary impact of these procedures, we observe heterogeneity of the effect of these procedures on measures of fairness in calibration and ranking across experimental conditions. Beyond the reported trade-offs, we emphasize that analyses of algorithmic fairness in healthcare lack the contextual grounding and causal awareness necessary to reason about the mechanisms that lead to health disparities, as well as about the potential of algorithmic fairness methods to counteract those mechanisms. In light of these limitations, we encourage researchers building predictive models for clinical use to step outside the algorithmic fairness frame and engage critically with the broader sociotechnical context surrounding the use of machine learning in healthcare.

Published in the Journal of Biomedical Informatics (https://doi.org/10.1016/j.jbi.2020.103621). Version 3 updates acknowledgements and fixes typos

References in corpus (16)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Equality of Opportunity in Supervised Learning
• Inherent Trade-Offs in the Fair Determination of Risk Scores
• Towards a Critical Race Methodology in Algorithmic Fairness
• On Fairness and Calibration
• On the (im)possibility of fairness
• Avoiding Discrimination through Causal Reasoning
• Identifying and Correcting Label Bias in Machine Learning
• On integral probability metrics, ϕ-divergences and binary classification
• A new paradigm for accelerating clinical data science at Stanford Medicine
• The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric
• Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics
• "The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning
• Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Context
• Causal Modeling for Fairness in Dynamical Systems
• Understanding racial bias in health using the Medical Expenditure Panel Survey data

Cited by in corpus (10)

• A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models
• Medical Imaging and Machine Learning
• Socially Responsible AI Algorithms: Issues, Purposes, and Challenges
• Bias Discovery in Machine Learning Models for Mental Health
• RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR
• Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings
• Mitigating Health Disparities in EHR via Deconfounder
• (Un)fairness in Post-operative Complication Prediction Models
• FERI: A Multitask-based Fairness Achieving Algorithm with Applications to Fair Organ Transplantation
• An Empirical Framework for Domain Generalization in Clinical Settings