Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing

Abstract

Machine learning (ML) holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. An important step is to characterize the (un)fairness of ML models - their tendency to perform differently across subgroups of the population - and to understand its underlying mechanisms. One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data. However, diagnosing this phenomenon is difficult, especially when sensitive attributes are causally linked with disease. Using multi-task learning, we propose the first method to assess and mitigate shortcut learning as a part of the fairness assessment of clinical ML systems, and demonstrate its application to clinical tasks in radiology and dermatology. Finally, our approach reveals instances when shortcutting is not responsible for unfairness, highlighting the need for a holistic approach to fairness mitigation in medical AI.

References in corpus (9)

• Array Programming with NumPy
• Reading Race: AI Recognises Patient's Racial Identity In Medical Images
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• Does "AI" stand for augmenting inequality in the era of covid-19 healthcare?
• This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
• Just Train Twice: Improving Group Robustness without Training Group Information
• Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendations
• Diagnosing failures of fairness transfer across distribution shift in real-world medical settings
• Causally motivated Shortcut Removal Using Auxiliary Labels

Cited by in corpus (6)

• Towards objective and systematic evaluation of bias in artificial intelligence for medical imaging
• Clinical Domain Knowledge-Derived Template Improves Post Hoc AI Explanations in Pneumothorax Classification
• How You Split Matters: Data Leakage and Subject Characteristics Studies in Longitudinal Brain MRI Analysis
• Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methods
• Integrating Deep Learning with Fundus and Optical Coherence Tomography for Cardiovascular Disease Prediction
• Correct-By-Construction: Certified Individual Fairness through Neural Network Training