Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
arXiv:2110.14019
Abstract
Unpredictable ML model behavior on unseen data, especially in the health domain, raises serious concerns about its safety as repercussions for mistakes can be fatal. In this paper, we explore the feasibility of using state-of-the-art out-of-distribution detectors for reliable and trustworthy diagnostic predictions. We select publicly available deep learning models relating to various health conditions (e.g., skin cancer, lung sound, and Parkinson's disease) using various input data types (e.g., image, audio, and motion data). We demonstrate that these models show unreasonable predictions on out-of-distribution datasets. We show that Mahalanobis distance- and Gram matrices-based out-of-distribution detection methods are able to detect out-of-distribution data with high accuracy for the health models that operate on different modalities. We then translate the out-of-distribution score into a human interpretable CONFIDENCE SCORE to investigate its effect on the users' interaction with health ML applications. Our user study shows that the \textsc{confidence score} helped the participants only trust the results with a high score to make a medical decision and disregard results with a low score. Through this work, we demonstrate that dataset shift is a critical piece of information for high-stake ML applications, such as medical diagnosis and healthcare, to provide reliable and trustworthy predictions to the users.
arXiv commentNeu
References in corpus (19)
- Deep Learning with Differential Privacy
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
- Sanity Checks for Saliency Maps
- Deep Anomaly Detection with Outlier Exposure
- Predictive Uncertainty Estimation via Prior Networks
- Mobile Sensor Data Anonymization
- Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement
- Adversarial Examples Are a Natural Consequence of Test Error in Noise
- Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
- Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
- Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
- A Benchmark of Medical Out of Distribution Detection
- Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
- Calibrating Healthcare AI: Towards Reliable and Interpretable Deep Predictive Models
- FRODO: Free rejection of out-of-distribution samples: application to chest x-ray analysis
- Security and Machine Learning in the Real World
- Self-Supervised Out-of-Distribution Detection in Brain CT Scans
- Smartphone Camera Oximetry in an Induced Hypoxemia Study