RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning

Abstract

We introduce a new semi-supervised, time series anomaly detection algorithm that uses deep reinforcement learning (DRL) and active learning to efficiently learn and adapt to anomalies in real-world time series data. Our model - called RLAD - makes no assumption about the underlying mechanism that produces the observation sequence and continuously adapts the detection model based on experience with anomalous patterns. In addition, it requires no manual tuning of parameters and outperforms all state-of-art methods we compare with, both unsupervised and semi-supervised, across several figures of merit. More specifically, we outperform the best unsupervised approach by a factor of 1.58 on the F1 score, with only 1% of labels and up to around 4.4x on another real-world dataset with only 0.1% of labels. We compare RLAD with seven deep-learning based algorithms across two common anomaly detection datasets with up to around 3M data points and between 0.28% to 2.65% anomalies.We outperform all of them across several important performance metrics.

References in corpus (6)

• Semi-Supervised Learning with Deep Generative Models
• Time-Series Anomaly Detection Service at Microsoft
• Deep Reinforcement Learning: An Overview
• Toward Supervised Anomaly Detection
• Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection
• Deep Semi-Supervised Anomaly Detection

Cited by in corpus (4)

• Expert enhanced dynamic time warping based anomaly detection
• Tackling Diverse Minorities in Imbalanced Classification
• Infrastructure-based End-to-End Learning and Prevention of Driver Failure
• A Sketch Based Game Theoretic Approach to Detect Anomalous Dense Sub-Communities in Large Data Streams