Time-Series Anomaly Detection Service at Microsoft

Abstract

Large companies need to monitor various metrics (for example, Page Views and Revenue) of their applications and services in real time. At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. In this paper, we introduce the pipeline and algorithm of our anomaly detection service, which is designed to be accurate, efficient and general. The pipeline consists of three major modules, including data ingestion, experimentation platform and online compute. To tackle the problem of time-series anomaly detection, we propose a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). Our work is the first attempt to borrow the SR model from visual saliency detection domain to time-series anomaly detection. Moreover, we innovatively combine SR and CNN together to improve the performance of SR model. Our approach achieves superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.

KDD 2019

Cited by in corpus (17)

• An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
• Adaptive Performance Anomaly Detection for Online Service Systems via Pattern Sketching
• Multivariate Time-series Anomaly Detection via Graph Attention Network
• RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning
• Sintel: A Machine Learning Framework to Extract Insights from Signals
• Merlion: A Machine Learning Library for Time Series
• Detecting Anomalies within Smart Buildings using Do-It-Yourself Internet of Things
• NVAE-GAN Based Approach for Unsupervised Time Series Anomaly Detection
• Series Saliency: Temporal Interpretation for Multivariate Time Series Forecasting
• Anomaly Detection on IT Operation Series via Online Matrix Profile
• Anomaly Detection on Seasonal Metrics via Robust Time Series Decomposition
• Statistical Evaluation of Anomaly Detectors for Sequences
• Automated Model Selection for Time-Series Anomaly Detection
• Lumos: A Library for Diagnosing Metric Regressions in Web-Scale Applications
• Fast Partial Fourier Transform
• Detecting Anomalous LAN Activities under Differential Privacy
• Comprehensible Counterfactual Explanation on Kolmogorov-Smirnov Test