Generalized Out-of-Distribution Detection: A Survey

Abstract

Out-of-distribution (OOD) detection is critical to ensuring the reliability and safety of machine learning systems. For instance, in autonomous driving, we would like the driving system to issue an alert and hand over the control to humans when it detects unusual scenes or objects that it has never seen during training time and cannot make a safe decision. The term, OOD detection, first emerged in 2017 and since then has received increasing attention from the research community, leading to a plethora of methods developed, ranging from classification-based to density-based to distance-based ones. Meanwhile, several other problems, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD), are closely related to OOD detection in terms of motivation and methodology. Despite common goals, these topics develop in isolation, and their subtle differences in definition and problem setting often confuse readers and practitioners. In this survey, we first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i.e., AD, ND, OSR, OOD detection, and OD. Under our framework, these five problems can be seen as special cases or sub-tasks, and are easier to distinguish. We then review each of these five areas by summarizing their recent technical developments, with a special focus on OOD detection methodologies. We conclude this survey with open challenges and potential research directions.

Feel free to comment on our Overleaf manuscript: https://www.overleaf.com/9899719915wmccvdtwpkct#c25192

References in corpus (22)

• Explaining and Harnessing Adversarial Examples
• Learning Transferable Visual Models From Natural Language Supervision
• Improved Regularization of Convolutional Neural Networks with Cutout
• Deep Learning for Anomaly Detection: A Survey
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Learning Open Set Network with Discriminative Reciprocal Points
• The Deepfake Detection Challenge (DFDC) Preview Dataset
• Exploring the Limits of Out-of-Distribution Detection
• CutPaste: Self-Supervised Learning for Anomaly Detection and Localization
• A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
• Open-World Semi-Supervised Learning
• A Survey on Open Set Recognition
• Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
• 'In-Between' Uncertainty in Bayesian Neural Networks
• Feature Space Singularity for Out-of-Distribution Detection
• ML-LOO: Detecting Adversarial Examples with Feature Attribution
• Novelty Detection Via Blurring
• One-vs-Rest Network-based Deep Probability Model for Open Set Recognition
• M2IOSR: Maximal Mutual Information Open Set Recognition
• MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space
• DeeperForensics Challenge 2020 on Real-World Face Forgery Detection: Methods and Results
• Few-Shot Open-Set Recognition using Meta-Learning

Cited by in corpus (29)

• Domain Generalization: A Survey
• A Review of Generalized Zero-Shot Learning Methods
• Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification
• Embrace Limited and Imperfect Training Datasets: Opportunities and Challenges in Plant Disease Recognition Using Deep Learning
• Can multi-label classification networks know what they don't know?
• Learning Discriminative Representations and Decision Boundaries for Open Intent Detection
• Towards out of distribution generalization for problems in mechanics
• Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes
• Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness
• Statistical process monitoring of artificial neural networks
• Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation
• Unified Classification and Rejection: A One-versus-All Framework
• Identifying the Defective: Detecting Damaged Grains for Cereal Appearance Inspection
• Open-World Lifelong Graph Learning
• ReSmooth: Detecting and Utilizing OOD Samples when Training with Data Augmentation
• Cycle Consistency-based Uncertainty Quantification of Neural Networks in Inverse Imaging Problems
• Simulating high-time resolution radio-telescope observations
• torchosr -- a PyTorch extension package for Open Set Recognition models evaluation in Python
• Distributed Harmonization: Federated Clustered Batch Effect Adjustment and Generalization
• Detecting Out-of-distribution Objects Using Neuron Activation Patterns
• A Multi-Scale Framework for Out-of-Distribution Detection in Dermoscopic Images
• Out-of-Distribution Detection for Adaptive Computer Vision
• Single-Entity Spiking Neuron Models: Survey
• EXOT: Exit-aware Object Tracker for Safe Robotic Manipulation of Moving Object
• Familiarity-Based Open-Set Recognition Under Adversarial Attacks
• BED: Bi-Encoder-Based Detectors for Out-of-Distribution Detection
• Biquality Learning: a Framework to Design Algorithms Dealing with Closed-Set Distribution Shifts
• On the detection of Out-Of-Distribution samples in Multiple Instance Learning
• A benchmark with decomposed distribution shifts for 360 monocular depth estimation