Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection
arXiv:2012.05329
Abstract
A crucial requirement for reliable deployment of deep learning models for safety-critical applications is the ability to identify out-of-distribution (OOD) data points, samples which differ from the training data and on which a model might underperform. Previous work has attempted to tackle this problem using uncertainty estimation techniques. However, there is empirical evidence that a large family of these techniques do not detect OOD reliably in classification tasks. This paper gives a theoretical explanation for said experimental findings and illustrates it on synthetic data. We prove that such techniques are not able to reliably identify OOD samples in a classification setting, since their level of confidence is generalized to unseen areas of the feature space. This result stems from the interplay between the representation of ReLU networks as piece-wise affine transformations, the saturating nature of activation functions like softmax, and the most widely-used uncertainty metrics.
References in corpus (12)
- Scikit-learn: Machine Learning in Python
- Gaussian Error Linear Units (GELUs)
- On Calibration of Modern Neural Networks
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Weight Uncertainty in Neural Networks
- From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
- On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning
- An Exploration of Softmax Alternatives Belonging to the Spherical Loss Family
- Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts
- On Controllable Sparse Alternatives to Softmax
- Well-calibrated Model Uncertainty with Temperature Scaling for Dropout Variational Inference
- Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures