A Joint Model for IT Operation Series Prediction and Anomaly Detection

Abstract

Status prediction and anomaly detection are two fundamental tasks in automatic IT systems monitoring. In this paper, a joint model Predictor & Anomaly Detector (PAD) is proposed to address these two issues under one framework. In our design, the variational auto-encoder (VAE) and long short-term memory (LSTM) are joined together. The prediction block (LSTM) takes clean input from the reconstructed time series by VAE, which makes it robust to the anomalies and noise for prediction task. In the meantime, the LSTM block maintains the long-term sequential patterns, which are out of the sight of a VAE encoding window. This leads to the better performance of VAE in anomaly detection than it is trained alone. In the whole processing pipeline, the spectral residual analysis is integrated with VAE and LSTM to boost the performance of both. The superior performance on two tasks is confirmed with the experiments on two challenging evaluation benchmarks.

arXiv commentThis paper has been published in Neurocomputing

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• Auto-Encoding Variational Bayes
• Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
• Time-Series Anomaly Detection Service at Microsoft
• A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
• Deep and Confident Prediction for Time Series at Uber
• VELC: A New Variational AutoEncoder Based Model for Time Series Anomaly Detection

Cited by in corpus (4)

• Deep Learning for Time Series Anomaly Detection: A Survey
• An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
• Is Your Anomaly Detector Ready for Change? Adapting AIOps Solutions to the Real World
• Anomaly Detection on IT Operation Series via Online Matrix Profile