Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications

Abstract

To ensure undisrupted business, large Internet companies need to closely monitor various KPIs (e.g., Page Views, number of online users, and number of orders) of its Web applications, to accurately detect anomalies and trigger timely troubleshooting/mitigation. However, anomaly detection for these seasonal KPIs with various patterns and data quality has been a great challenge, especially without labels. In this paper, we proposed Donut, an unsupervised anomaly detection algorithm based on VAE. Thanks to a few of our key techniques, Donut greatly outperforms a state-of-arts supervised ensemble approach and a baseline VAE approach, and its best F-scores range from 0.75 to 0.9 for the studied KPIs from a top global Internet company. We come up with a novel KDE interpretation of reconstruction for Donut, making it the first VAE-based anomaly detection algorithm with solid theoretical explanation.

12 pages (including references), 17 figures, submitted to WWW 2018: The 2018 Web Conference, April 23--27, 2018, Lyon, France. The contents discarded from the conference version due to the 9-page limitation are also included in this version

References in corpus (1)

• Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications

Cited by in corpus (34)

• A Unifying Review of Deep and Shallow Anomaly Detection
• Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
• Time-Series Anomaly Detection Service at Microsoft
• Deep Learning for Time Series Anomaly Detection: A Survey
• DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection
• An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
• Feature Encoding with AutoEncoders for Weakly-supervised Anomaly Detection
• TFAD: A Decomposition Time Series Anomaly Detection Architecture with Time-Frequency Analysis
• DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications
• Correlation-aware Spatial-Temporal Graph Learning for Multivariate Time-series Anomaly Detection
• Time Series Anomaly Detection for Cyber-Physical Systems via Neural System Identification and Bayesian Filtering
• Local Evaluation of Time Series Anomaly Detection Algorithms
• Unsupervised Anomaly Detection in Time-series: An Extensive Evaluation and Analysis of State-of-the-art Methods
• Unsupervised Time Series Outlier Detection with Diversity-Driven Convolutional Ensembles -- Extended Version
• A Joint Model for IT Operation Series Prediction and Anomaly Detection
• Is it worth it? Comparing six deep and classical methods for unsupervised anomaly detection in time series
• Adaptive Performance Anomaly Detection for Online Service Systems via Pattern Sketching
• Anomaly Detection based on Zero-Shot Outlier Synthesis and Hierarchical Feature Distillation
• ReRe: A Lightweight Real-time Ready-to-Go Anomaly Detection Approach for Time Series
• StRegA: Unsupervised Anomaly Detection in Brain MRIs using a Compact Context-encoding Variational Autoencoder
• Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder
• NFAD: Fixing anomaly detection using normalizing flows
• Unsupervised Anomaly Detection with an Enhanced Teacher for Student-Teacher Feature Pyramid Matching
• A Machine-Learning Phase Classification Scheme for Anomaly Detection in Signals with Periodic Characteristics
• Landing AI on Networks: An equipment vendor viewpoint on Autonomous Driving Networks
• How Far Should We Look Back to Achieve Effective Real-Time Time-Series Anomaly Detection?
• AlerTiger: Deep Learning for AI Model Health Monitoring at LinkedIn
• Is Your Anomaly Detector Ready for Change? Adapting AIOps Solutions to the Real World
• Little Help Makes a Big Difference: Leveraging Active Learning to Improve Unsupervised Time Series Anomaly Detection
• Automated Contrastive Learning Strategy Search for Time Series
• xLSTMAD: A Powerful xLSTM-based Method for Anomaly Detection
• Robust Group Anomaly Detection for Quasi-Periodic Network Time Series
• Noise-Resilient Point-wise Anomaly Detection in Time Series Using Weak Segment Labels
• Learning Sparsity of Representations with Discrete Latent Variables