Detecting OODs as datapoints with High Uncertainty
arXiv:2108.06380
Abstract
Deep neural networks (DNNs) are known to produce incorrect predictions with very high confidence on out-of-distribution inputs (OODs). This limitation is one of the key challenges in the adoption of DNNs in high-assurance systems such as autonomous driving, air traffic management, and medical diagnosis. This challenge has received significant attention recently, and several techniques have been developed to detect inputs where the model's prediction cannot be trusted. These techniques detect OODs as datapoints with either high epistemic uncertainty or high aleatoric uncertainty. We demonstrate the difference in the detection ability of these techniques and propose an ensemble approach for detection of OODs as datapoints with high uncertainty (epistemic or aleatoric). We perform experiments on vision datasets with multiple DNN architectures, achieving state-of-the-art results in most cases.
arXiv admin note: text overlap with arXiv:2103.12628
References in corpus (11)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- End to End Learning for Self-Driving Cars
- On Calibration of Modern Neural Networks
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
- Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
- Deep Anomaly Detection with Outlier Exposure
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
- Classification-Based Anomaly Detection for General Data
- Unsupervised Risk Estimation Using Only Conditional Independence Structure