Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networks

Abstract

We investigate anomaly detection in an unsupervised framework and introduce Long Short Term Memory (LSTM) neural network based algorithms. In particular, given variable length data sequences, we first pass these sequences through our LSTM based structure and obtain fixed length sequences. We then find a decision function for our anomaly detectors based on the One Class Support Vector Machines (OC-SVM) and Support Vector Data Description (SVDD) algorithms. As the first time in the literature, we jointly train and optimize the parameters of the LSTM architecture and the OC-SVM (or SVDD) algorithm using highly effective gradient and quadratic programming based training methods. To apply the gradient based training method, we modify the original objective criteria of the OC-SVM and SVDD algorithms, where we prove the convergence of the modified objective criteria to the original criteria. We also provide extensions of our unsupervised formulation to the semi-supervised and fully supervised frameworks. Thus, we obtain anomaly detection algorithms that can process variable length data sequences while providing high performance, especially for time series data. Our approach is generic so that we also apply this approach to the Gated Recurrent Unit (GRU) architecture by directly replacing our LSTM based structure with the GRU based structure. In our experiments, we illustrate significant performance gains achieved by our algorithms with respect to the conventional methods.

References in corpus (7)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Full-Capacity Unitary Recurrent Neural Networks
• Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences
• Optimal Index Policies for Anomaly Localization in Resource-Constrained Cyber Systems
• Gated Orthogonal Recurrent Units: On Learning to Forget
• Tensorial Recurrent Neural Networks for Longitudinal Data Analysis
• Fisher Information Framework for Time Series Modeling

Cited by in corpus (17)

• Deep Learning for Anomaly Detection: A Survey
• Deep Learning for Time Series Anomaly Detection: A Survey
• A Survey on Anomaly Detection for Technical Systems using LSTM Networks
• Feature Encoding with AutoEncoders for Weakly-supervised Anomaly Detection
• DTAAD: Dual Tcn-Attention Networks for Anomaly Detection in Multivariate Time Series Data
• Unsupervised Anomaly Detection in Stream Data with Online Evolving Spiking Neural Networks
• Deep Semi-Supervised Anomaly Detection
• An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contamination
• Hybridization of Capsule and LSTM Networks for unsupervised anomaly detection on multivariate data
• DyEdgeGAT: Dynamic Edge via Graph Attention for Early Fault Detection in IIoT Systems
• Credit card fraud detection using machine learning: A survey
• ESAD: End-to-end Deep Semi-supervised Anomaly Detection
• AlerTiger: Deep Learning for AI Model Health Monitoring at LinkedIn
• A Combination of Temporal Sequence Learning and Data Description for Anomaly-based NIDS
• Extending Isolation Forest for Anomaly Detection in Big Data via K-Means
• Deep Variational Semi-Supervised Novelty Detection
• Unsupervised detection and open-set classification of fast-ramped flexibility activation events