Are all outliers alike? On Understanding the Diversity of Outliers for Detecting OODs
arXiv:2103.12628
Abstract
Deep neural networks (DNNs) are known to produce incorrect predictions with very high confidence on out-of-distribution (OOD) inputs. This limitation is one of the key challenges in the adoption of deep learning models in high-assurance systems such as autonomous driving, air traffic management, and medical diagnosis. This challenge has received significant attention recently, and several techniques have been developed to detect inputs where the model's prediction cannot be trusted. These techniques use different statistical, geometric, or topological signatures. This paper presents a taxonomy of OOD outlier inputs based on their source and nature of uncertainty. We demonstrate how different existing detection approaches fail to detect certain types of outliers. We utilize these insights to develop a novel integrated detection approach that uses multiple attributes corresponding to different types of outliers. Our results include experiments on CIFAR10, SVHN and MNIST as in-distribution data and Imagenet, LSUN, SVHN (for CIFAR10), CIFAR10 (for SVHN), KMNIST, and F-MNIST as OOD data across different DNN architectures such as ResNet34, WideResNet, DenseNet, and LeNet5.
References in corpus (10)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- End to End Learning for Self-Driving Cars
- Wide Residual Networks
- On Calibration of Modern Neural Networks
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
- Deep Anomaly Detection with Outlier Exposure
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
- Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples