Is it worth it? Comparing six deep and classical methods for unsupervised anomaly detection in time series

Abstract

Detecting anomalies in time series data is important in a variety of fields, including system monitoring, healthcare, and cybersecurity. While the abundance of available methods makes it difficult to choose the most appropriate method for a given application, each method has its strengths in detecting certain types of anomalies. In this study, we compare six unsupervised anomaly detection methods of varying complexity to determine whether more complex methods generally perform better and if certain methods are better suited to certain types of anomalies. We evaluated the methods using the UCR anomaly archive, a recent benchmark dataset for anomaly detection. We analyzed the results on a dataset and anomaly type level after adjusting the necessary hyperparameters for each method. Additionally, we assessed the ability of each method to incorporate prior knowledge about anomalies and examined the differences between point-wise and sequence-wise features. Our experiments show that classical machine learning methods generally outperform deep learning methods across a range of anomaly types.

arXiv comment17 Pages, The repository to reproduce the results is available at https://gitlab.com/dlr-dw/is-it-worth-it-benchmark

References in corpus (8)

• Deep Learning for Anomaly Detection: A Review
• Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
• A Unifying Review of Deep and Shallow Anomaly Detection
• Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
• Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark
• Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress
• Detecting Regions of Maximal Divergence for Spatio-Temporal Anomaly Detection
• A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges

Cited by in corpus (5)

• Explainable Time Series Anomaly Detection using Masked Latent Generative Modeling
• DeepHYDRA: Resource-Efficient Time-Series Anomaly Detection in Dynamically-Configured Systems
• Wavelet-based temporal models of human activity for anomaly detection in smart robot-assisted environments
• PATH: A Discrete-sequence Dataset for Evaluating Online Unsupervised Anomaly Detection Approaches for Multivariate Time Series
• Unraveling Anomalies in Time: Unsupervised Discovery and Isolation of Anomalous Behavior in Bio-regenerative Life Support System Telemetry