Open Set Medical Diagnosis

Abstract

Machine-learned diagnosis models have shown promise as medical aides but are trained under a closed-set assumption, i.e. that models will only encounter conditions on which they have been trained. However, it is practically infeasible to obtain sufficient training data for every human condition, and once deployed such models will invariably face previously unseen conditions. We frame machine-learned diagnosis as an open-set learning problem, and study how state-of-the-art approaches compare. Further, we extend our study to a setting where training data is distributed across several healthcare sites that do not allow data pooling, and experiment with different strategies of building open-set diagnostic ensembles. Across both settings, we observe consistent gains from explicitly modeling unseen conditions, but find the optimal training strategy to vary across settings.

arXiv commentAbbreviated version to appear at Machine Learning for Healthcare (ML4H) Workshop at NeurIPS 2019

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
• Reducing Network Agnostophobia
• FADL:Federated-Autonomous Deep Learning for Distributed Electronic Health Record

Cited by in corpus (3)

• No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets
• The accuracy vs. coverage trade-off in patient-facing diagnosis models
• CheXseen: Unseen Disease Detection for Deep Learning Interpretation of Chest X-rays