A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges

Abstract

Machine learning models often encounter samples that are diverged from the training distribution. Failure to recognize an out-of-distribution (OOD) sample, and consequently assign that sample to an in-class label significantly compromises the reliability of a model. The problem has gained significant attention due to its importance for safety deploying models in open-world settings. Detecting OOD samples is challenging due to the intractability of modeling all possible unknown distributions. To date, several research domains tackle the problem of detecting unfamiliar samples, including anomaly detection, novelty detection, one-class learning, open set recognition, and out-of-distribution detection. Despite having similar and shared concepts, out-of-distribution, open-set, and anomaly detection have been investigated independently. Accordingly, these research avenues have not cross-pollinated, creating research barriers. While some surveys intend to provide an overview of these approaches, they seem to only focus on a specific domain without examining the relationship between different domains. This survey aims to provide a cross-domain and comprehensive review of numerous eminent works in respective areas while identifying their commonalities. Researchers can benefit from the overview of research advances in different fields and develop future methodology synergistically. Furthermore, to the best of our knowledge, while there are surveys in anomaly detection or one-class learning, there is no comprehensive or up-to-date survey on out-of-distribution detection, which our survey covers extensively. Finally, having a unified cross-domain perspective, we discuss and shed light on future lines of research, intending to bring these fields closer together.

Published in Transaction on Machine Learning (TMLR)

References in corpus (20)

• Distilling the Knowledge in a Neural Network
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Improved Regularization of Convolutional Neural Networks with Cutout
• NICE: Non-linear Independent Components Estimation
• CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
• Deep Learning for Anomaly Detection: A Survey
• SmoothGrad: removing noise by adding noise
• Deep Anomaly Detection with Outlier Exposure
• Learning Open Set Network with Discriminative Reciprocal Points
• Exploring the Limits of Out-of-Distribution Detection
• One-Class Classification: A Survey
• Classification-Based Anomaly Detection for General Data
• Unsolved Problems in ML Safety
• CutPaste: Self-Supervised Learning for Anomaly Detection and Localization
• Reducing Network Agnostophobia
• A PCB Dataset for Defects Detection and Classification
• A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
• SSD: A Unified Framework for Self-Supervised Outlier Detection
• Understanding the Effect of Bias in Deep Anomaly Detection
• Provably Robust Detection of Out-of-distribution Data (almost) for free

Cited by in corpus (3)

• Is it worth it? Comparing six deep and classical methods for unsupervised anomaly detection in time series
• CL2R: Compatible Lifelong Learning Representations
• Toward a Realistic Benchmark for Out-of-Distribution Detection