Benchmarking datasets for Anomaly-based Network Intrusion Detection: KDD CUP 99 alternatives

Abstract

Machine Learning has been steadily gaining traction for its use in Anomaly-based Network Intrusion Detection Systems (A-NIDS). Research into this domain is frequently performed using the KDD~CUP~99 dataset as a benchmark. Several studies question its usability while constructing a contemporary NIDS, due to the skewed response distribution, non-stationarity, and failure to incorporate modern attacks. In this paper, we compare the performance for KDD-99 alternatives when trained using classification models commonly found in literature: Neural Network, Support Vector Machine, Decision Tree, Random Forest, Naive Bayes and K-Means. Applying the SMOTE oversampling technique and random undersampling, we create a balanced version of NSL-KDD and prove that skewed target classes in KDD-99 and NSL-KDD hamper the efficacy of classifiers on minority classes (U2R and R2L), leading to possible security risks. We explore UNSW-NB15, a modern substitute to KDD-99 with greater uniformity of pattern distribution. We benchmark this dataset before and after SMOTE oversampling to observe the effect on minority performance. Our results indicate that classifiers trained on UNSW-NB15 match or better the Weighted F1-Score of those trained on NSL-KDD and KDD-99 in the binary case, thus advocating UNSW-NB15 as a modern substitute to these datasets.

Paper accepted into Proceedings of IEEE International Conference on Computing, Communication and Security 2018 (ICCCS-2018) Statistics: 8 pages, 7 tables, 3 figures, 34 references

References in corpus (3)

• Scikit-learn: Machine Learning in Python
• SMOTE: Synthetic Minority Over-sampling Technique
• Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning

Cited by in corpus (9)

• The Cross-evaluation of Machine Learning-based Network Intrusion Detection Systems
• Anomaly-Based Intrusion Detection by Machine Learning: A Case Study on Probing Attacks to an Institutional Network
• Data-Driven Network Intrusion Detection: A Taxonomy of Challenges and Methods
• ScarceGAN: Discriminative Classification Framework for Rare Class Identification for Longitudinal Data with Weak Prior
• Explainable and Interpretable Forecasts on Non-Smooth Multivariate Time Series for Responsible Gameplay
• An Investigation into the Performance of Non-Contrastive Self-Supervised Learning Methods for Network Intrusion Detection
• Analyzing Root Causes of Intrusion Detection False-Negatives: Methodology and Case Study
• Impact of Data Snooping on Deep Learning Models for Locating Vulnerabilities in Lifted Code
• Improving the Reliability of Network Intrusion Detection Systems through Dataset Integration