Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity

Abstract

The ability to detect out-of-distribution (OOD) samples is vital to secure the reliability of deep neural networks in real-world applications. Considering the nature of OOD samples, detection methods should not have hyperparameters that need to be tuned depending on incoming OOD samples. However, most of the recently proposed methods do not meet this requirement, leading to compromised performance in real-world applications. In this paper, we propose a simple, hyperparameter-free method based on softmax of scaled cosine similarity. It resembles the approach employed by modern metric learning methods, but it differs in details; the differences are essential to achieve high detection performance. We show through experiments that our method outperforms the existing methods on the evaluation test recently proposed by Shafaei et al., which takes the above issue of hyperparameter dependency into account. We also show that it achieves at least comparable performance to other methods on the conventional test, where their hyperparameters are chosen using explicit OOD samples. Furthermore, it is computationally more efficient than most of the previous methods, since it needs only a single forward pass.

arXiv commentExtend the supplementary material

References in corpus (19)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distilling the Knowledge in a Neural Network
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Deep Anomaly Detection with Outlier Exposure
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• Accurate Uncertainties for Deep Learning Using Calibrated Regression
• TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking
• Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Distance-based Confidence Score for Neural Network Classifiers
• Leveraging Uncertainty Estimates for Predicting Segmentation Quality
• Confidence estimation in Deep Neural networks via density modelling
• Robustly representing uncertainty in deep neural networks through sampling

Cited by in corpus (5)

• Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data
• Anomalous Example Detection in Deep Learning: A Survey
• Practical Evaluation of Out-of-Distribution Detection Methods for Image Classification
• Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data
• Bridging In- and Out-of-distribution Samples for Their Better Discriminability