A Critical Review of Common Log Data Sets Used for Evaluation of Sequence-based Anomaly Detection Techniques

Abstract

Log data store event execution patterns that correspond to underlying workflows of systems or applications. While most logs are informative, log data also include artifacts that indicate failures or incidents. Accordingly, log data are often used to evaluate anomaly detection techniques that aim to automatically disclose unexpected or otherwise relevant system behavior patterns. Recently, detection approaches leveraging deep learning have increasingly focused on anomalies that manifest as changes of sequential patterns within otherwise normal event traces. Several publicly available data sets, such as HDFS, BGL, Thunderbird, OpenStack, and Hadoop, have since become standards for evaluating these anomaly detection techniques, however, the appropriateness of these data sets has not been closely investigated in the past. In this paper we therefore analyze six publicly available log data sets with focus on the manifestations of anomalies and simple techniques for their detection. Our findings suggest that most anomalies are not directly related to sequential manifestations and that advanced detection techniques are not required to achieve high detection rates on these data sets.

References in corpus (5)

• Log-based Anomaly Detection with Deep Learning: How Far Are We?
• Deep Learning for Anomaly Detection in Log Data: A Survey
• Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection
• Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics
• Semi-supervised learning via DQN for log anomaly detection

Cited by in corpus (1)

• A Comprehensive Study of Machine Learning Techniques for Log-Based Anomaly Detection