FRODO: Free rejection of out-of-distribution samples: application to chest x-ray analysis

Abstract

In this work, we propose a method to reject out-of-distribution samples which can be adapted to any network architecture and requires no additional training data. Publicly available chest x-ray data (38,353 images) is used to train a standard ResNet-50 model to detect emphysema. Feature activations of intermediate layers are used as descriptors defining the training data distribution. A novel metric, FRODO, is measured by using the Mahalanobis distance of a new test sample to the training data distribution. The method is tested using a held-out test dataset of 21,176 chest x-rays (in-distribution) and a set of 14,821 out-of-distribution x-ray images of incorrect orientation or anatomy. In classifying test samples as in or out-of distribution, our method achieves an AUC score of 0.99.

arXiv commentMIDL 2019 [arXiv:1907.08612]

References in corpus (5)

• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Opening the Black Box of Deep Neural Networks via Information
• Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance

Cited by in corpus (5)

• Deep Learning for Chest X-ray Analysis: A Survey
• Recommendations on test datasets for evaluating AI solutions in pathology
• Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
• Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
• A versatile anomaly detection method for medical images with a flow-based generative model in semi-supervision setting