Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation

Abstract

In our digital universe nowadays, enormous amount of data are produced in a streaming manner in a variety of application areas. These data are often unlabelled. In this case, identifying infrequent events, such as anomalies, poses a great challenge. This problem becomes even more difficult in non-stationary environments, which can cause deterioration of the predictive performance of a model. To address the above challenges, the paper proposes an autoencoder-based incremental learning method with drift detection (strAEm++DD). Our proposed method strAEm++DD leverages on the advantages of both incremental learning and drift detection. We conduct an experimental study using real-world and synthetic datasets with severe or extreme class imbalance, and provide an empirical analysis of strAEm++DD. We further conduct a comparative study, showing that the proposed method significantly outperforms existing baseline and advanced methods.

anomaly detection, concept drift, incremental anomaly detection, concept drift, incremental learning, autoencoders, data streams, class imbalance, nonstationary environments

References in corpus (4)

• Deep Learning for Anomaly Detection: A Survey
• Online Learning With Adaptive Rebalancing in Nonstationary Environments
• Nonstationary data stream classification with online active learning and siamese neural networks
• Data-efficient Online Classification with Siamese Networks and Active Learning

Cited by in corpus (4)

• Unsupervised Incremental Learning with Dual Concept Drift Detection for Identifying Anomalous Sequences
• SiameseDuo++: Active Learning from Data Streams with Dual Augmented Siamese Networks
• Online Detection of Water Contamination Under Concept Drift
• Incremental Learning with Concept Drift Detection and Prototype-based Embeddings for Graph Stream Classification