Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data

Abstract

Deep neural networks have attained remarkable performance when applied to data that comes from the same distribution as that of the training set, but can significantly degrade otherwise. Therefore, detecting whether an example is out-of-distribution (OoD) is crucial to enable a system that can reject such samples or alert users. Recent works have made significant progress on OoD benchmarks consisting of small image datasets. However, many recent methods based on neural networks rely on training or tuning with both in-distribution and out-of-distribution data. The latter is generally hard to define a-priori, and its selection can easily bias the learning. We base our work on a popular method ODIN, proposing two strategies for freeing it from the needs of tuning with OoD data, while improving its OoD detection performance. We specifically propose to decompose confidence scoring as well as a modified input pre-processing method. We show that both of these significantly help in detection performance. Our further analysis on a larger scale image dataset shows that the two types of distribution shifts, specifically semantic shift and non-semantic shift, present a significant difference in the difficulty of the problem, providing an analysis of when ODIN-like strategies do or do not work.

arXiv commentCVPR 2020

References in corpus (9)

• Distilling the Knowledge in a Neural Network
• Wide Residual Networks
• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Deep Anomaly Detection with Outlier Exposure
• Reducing Network Agnostophobia
• Metric Learning for Novelty and Anomaly Detection
• Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity

Cited by in corpus (5)

• Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP
• Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
• SLA${}^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation
• On Out-of-distribution Detection with Energy-based Models
• Learning to Cascade: Confidence Calibration for Improving the Accuracy and Computational Cost of Cascade Inference Systems