A Unifying Review of Deep and Shallow Anomaly Detection

Abstract

Deep learning approaches to anomaly detection have recently improved the state of the art in detection performance on complex datasets such as large collections of images or text. These results have sparked a renewed interest in the anomaly detection problem and led to the introduction of a great variety of new methods. With the emergence of numerous such methods, including approaches based on generative models, one-class classification, and reconstruction, there is a growing need to bring methods of this field into a systematic and unified perspective. In this review we aim to identify the common underlying principles as well as the assumptions that are often made implicitly by various methods. In particular, we draw connections between classic 'shallow' and novel deep approaches and show how this relation might cross-fertilize or extend both directions. We further provide an empirical assessment of major existing methods that is enriched by the use of recent explainability techniques, and present specific worked-through examples together with practical advice. Finally, we outline critical open challenges and identify specific paths for future research in anomaly detection.

40 pages; accepted for publication in the Proceedings of the IEEE;

References in corpus (37)

• Deep Learning in Neural Networks: An Overview
• A Survey on Deep Learning in Medical Image Analysis
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• Methods for Interpreting and Understanding Deep Neural Networks
• An Introduction to Variational Autoencoders
• Ensemble Adversarial Training: Attacks and Defenses
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Normalizing Flows: An Introduction and Review of Current Methods
• Searching for Exotic Particles in High-Energy Physics with Deep Learning
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
• Snorkel: Rapid Training Data Creation with Weak Supervision
• One-Class Classification: Taxonomy of Study and Review of Techniques
• Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark
• Toward Supervised Anomaly Detection
• Anomaly Detection with Density Estimation
• Learning Deep Features for One-Class Classification
• Anomaly Detection using Autoencoders in High Performance Computing Systems
• Unsupervised speech representation learning using WaveNet autoencoders
• Exploiting Epistemic Uncertainty of Anatomy Segmentation for Anomaly Detection in Retinal OCT
• One-Class Convolutional Neural Network
• Cascaded Region-based Densely Connected Network for Event Detection: A Seismic Application
• Unsupervised Detection of Anomalous Sound based on Deep Learning and the Neyman-Pearson Lemma
• Input complexity and out-of-distribution detection with likelihood-based generative models
• Variational Autoencoders for New Physics Mining at the Large Hadron Collider
• Optimal rates for plug-in estimators of density level sets
• Towards Explaining Anomalies: A Deep Taylor Decomposition of One-Class Models
• Deep Divergence-Based Approach to Clustering
• Safe Robot Navigation via Multi-Modal Anomaly Detection
• Finding outlier light-curves in catalogs of periodic variable stars
• Revealing quantum chaos with machine learning
• Simple and Effective Prevention of Mode Collapse in Deep One-Class Classification
• Fast Distance-based Anomaly Detection in Images Using an Inception-like Autoencoder

Cited by in corpus (73)

• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• A Comprehensive Survey on Graph Anomaly Detection with Deep Learning
• Deep Isolation Forest for Anomaly Detection
• DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection
• Deep Learning for Unsupervised Anomaly Localization in Industrial Images: A Survey
• Anomaly Detection-Inspired Few-Shot Medical Image Segmentation Through Self-Supervision With Supervoxels
• A Survey on Unsupervised Anomaly Detection Algorithms for Industrial Images
• Toward Explainable AI for Regression Models
• On the Nature and Types of Anomalies: A Review of Deviations in Data
• Autoencoders for unsupervised anomaly detection in high energy physics
• Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook
• Rule Extraction in Unsupervised Anomaly Detection for Model Explainability: Application to OneClass SVM
• Unsupervised Deep Learning for IoT Time Series
• Dual-distribution discrepancy with self-supervised refinement for anomaly detection in medical images
• Is it worth it? Comparing six deep and classical methods for unsupervised anomaly detection in time series
• Federated PCA on Grassmann Manifold for IoT Anomaly Detection
• The Clever Hans Effect in Unsupervised Learning
• Meta-survey on outlier and anomaly detection
• An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contamination
• SSDPT: Self-Supervised Dual-Path Transformer for Anomalous Sound Detection in Machine Condition Monitoring
• CARE to Compare: A real-world dataset for anomaly detection in wind turbine data
• Active anomaly detection based on deep one-class classification
• Learning image representations for anomaly detection: application to discovery of histological alterations in drug development
• Perfect density models cannot guarantee anomaly detection
• Anomaly Detection Based on Isolation Mechanisms: A Survey
• METER: A Dynamic Concept Adaptation Framework for Online Anomaly Detection
• Anomaly Detection with Score Distribution Discrimination
• ALGAN: Anomaly Detection by Generating Pseudo Anomalous Data via Latent Variables
• Robust online active learning
• An Outlier Exposure Approach to Improve Visual Anomaly Detection Performance for Mobile Robots
• Explainable AI guided unsupervised fault diagnostics for high-voltage circuit breakers
• Semisupervised Anomaly Detection using Support Vector Regression with Quantum Kernel
• Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset
• Deep Autoencoders for Anomaly Detection in Textured Images using CW-SSIM
• Multi-Perspective Anomaly Detection
• MarMot: Metamorphic Runtime Monitoring of Autonomous Driving Systems
• Hybrid Open-set Segmentation with Synthetic Negative Data
• OneFlow: One-class flow for anomaly detection based on a minimal volume region
• Fault Detection in New Wind Turbines with Limited Data by Generative Transfer Learning
• TracInAD: Measuring Influence for Anomaly Detection
• SAFE-OCC: A Novelty Detection Framework for Convolutional Neural Network Sensors and its Application in Process Control
• Double-Adversarial Activation Anomaly Detection: Adversarial Autoencoders are Anomaly Generators
• SPINEX: Similarity-based Predictions and Explainable Neighbors Exploration for Regression and Classification Tasks in Machine Learning
• Retrieval Augmented Deep Anomaly Detection for Tabular Data
• Sensing Anomalies as Potential Hazards: Datasets and Benchmarks
• Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection
• Quantum Autoencoder for Multivariate Time Series Anomaly Detection
• Diffusion-Based Adversarial Purification for Speaker Verification
• Constructing a meta-learner for unsupervised anomaly detection
• Anomaly-Aware YOLO: A Frugal yet Robust Approach to Infrared Small Target Detection
• SeMAnD: Self-Supervised Anomaly Detection in Multimodal Geospatial Datasets
• EntropyStop: Unsupervised Deep Outlier Detection with Loss Entropy
• Chromosomal Structural Abnormality Diagnosis by Homologous Similarity
• Understanding the limitations of self-supervised learning for tabular anomaly detection
• R2-AD2: Detecting Anomalies by Analysing the Raw Gradient
• ISP-AD: A Large-Scale Real-World Dataset for Advancing Industrial Anomaly Detection with Synthetic and Real Defects
• Visualization for Multivariate Gaussian Anomaly Detection in Images
• Leveraging the Christoffel Function for Outlier Detection in Data Streams
• Detecting Abnormal Operations in Concentrated Solar Power Plants from Irregular Sequences of Thermal Images
• Anomaly Detection by Recombining Gated Unsupervised Experts
• Reconstructing galaxy star formation histories from COSMOS2020 photometry using simulation-based inference
• Federated Attention Autoencoders with a Stochastic Aggregation Scheme for Anomaly Detection
• Identifying Astrophysical Anomalies in 99.6 Million Cutouts from the Hubble Legacy Archive Using AnomalyMatch
• Temporal Convolution-based Hybrid Model Approach with Representation Learning for Real-Time Acoustic Anomaly Detection
• Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity
• Critical Review for One-class Classification: recent advances and the reality behind them
• Circuit design in biology and machine learning. II. Anomaly detection
• A Multi-Source Ultrasound Benchmark Revealing the Limits of Contemporary Self-Supervised Anomaly Detection Methods
• Self-Supervised ConvLSTM for Fermi Large Area Telescope Transient Detection
• Unsupervised Deep One-Class Classification with Adaptive Threshold based on Training Dynamics
• Foundation models on the bridge: Semantic hazard detection and safety maneuvers for maritime autonomy with vision-language models
• Identifying Anomalous DESI Galaxy Spectra with a Variational Autoencoder