MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series

Abstract

In this paper, we introduce Masked Anomaly Detection (MAD), a general self-supervised learning task for multivariate time series anomaly detection. With the increasing availability of sensor data from industrial systems, being able to detecting anomalies from streams of multivariate time series data is of significant importance. Given the scarcity of anomalies in real-world applications, the majority of literature has been focusing on modeling normality. The learned normal representations can empower anomaly detection as the model has learned to capture certain key underlying data regularities. A typical formulation is to learn a predictive model, i.e., use a window of time series data to predict future data values. In this paper, we propose an alternative self-supervised learning task. By randomly masking a portion of the inputs and training a model to estimate them using the remaining ones, MAD is an improvement over the traditional left-to-right next step prediction (NSP) task. Our experimental results demonstrate that MAD can achieve better anomaly detection rates over traditional NSP approaches when using exactly the same neural network (NN) base models, and can be modified to run as fast as NSP models during test time on the same hardware, thus making it an ideal upgrade for many existing NSP-based NN anomaly detection models.

arXiv commentAccepted by the 2022 International Joint Conference on Neural Networks (IJCNN 2022)

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• A Simple Framework for Contrastive Learning of Visual Representations
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• WaveNet: A Generative Model for Raw Audio
• Deep Learning for Anomaly Detection: A Review
• Unsupervised Learning of Video Representations using LSTMs
• Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
• Unsupervised Representation Learning by Predicting Image Rotations
• Deep Learning for Anomaly Detection: A Survey
• Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
• Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case
• Unsupervised Learning by Predicting Noise
• Fault Detection and Identification using Bayesian Recurrent Neural Networks

Cited by in corpus (2)

• DyEdgeGAT: Dynamic Edge via Graph Attention for Early Fault Detection in IIoT Systems
• Hypergraph Learning based Recommender System for Anomaly Detection, Control and Optimization