Detecting Shortcuts in Medical Images -- A Case Study in Chest X-rays

Abstract

The availability of large public datasets and the increased amount of computing power have shifted the interest of the medical community to high-performance algorithms. However, little attention is paid to the quality of the data and their annotations. High performance on benchmark datasets may be reported without considering possible shortcuts or artifacts in the data, besides, models are not tested on subpopulation groups. With this work, we aim to raise awareness about shortcuts problems. We validate previous findings, and present a case study on chest X-rays using two publicly available datasets. We share annotations for a subset of pneumothorax images with drains. We conclude with general recommendations for medical image classification.

Submitted to ISBI 2023

References in corpus (2)

• How I failed machine learning in medical imaging -- shortcomings and recommendations
• Reading Race: AI Recognises Patient's Racial Identity In Medical Images

Cited by in corpus (8)

• In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
• How You Split Matters: Data Leakage and Subject Characteristics Studies in Longitudinal Brain MRI Analysis
• Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methods
• Dataset Distribution Impacts Model Fairness: Single vs. Multi-Task Learning
• Mask of truth: model sensitivity to unexpected regions of medical images
• Subgroup Performance Analysis in Hidden Stratifications
• meval: A Statistical Toolbox for Fine-Grained Model Performance Analysis
• Augmenting Chest X-ray Datasets with Non-Expert Annotations