NetFlow Datasets for Machine Learning-based Network Intrusion Detection Systems

Abstract

Machine Learning (ML)-based Network Intrusion Detection Systems (NIDSs) have proven to become a reliable intelligence tool to protect networks against cyberattacks. Network data features has a great impact on the performances of ML-based NIDSs. However, evaluating ML models often are not reliable, as each ML-enabled NIDS is trained and validated using different data features that may do not contain security events. Therefore, a common ground feature set from multiple datasets is required to evaluate an ML model's detection accuracy and its ability to generalise across datasets. This paper presents NetFlow features from four benchmark NIDS datasets known as UNSW-NB15, BoT-IoT, ToN-IoT, and CSE-CIC-IDS2018 using their publicly available packet capture files. In a real-world scenario, NetFlow features are relatively easier to extract from network traffic compared to the complex features used in the original datasets, as they are usually extracted from packet headers. The generated Netflow datasets have been labelled for solving binary- and multiclass-based learning challenges. Preliminary results indicate that NetFlow features lead to similar binary-class results and lower multi-class classification results amongst the four datasets compared to their respective original features datasets. The NetFlow datasets are named NF-UNSW-NB15, NF-BoT-IoT, NF-ToN-IoT, NF-CSE-CIC-IDS2018 and NF-UQ-NIDS are published at http://staff.itee.uq.edu.au/marius/NIDS_datasets/ for research purposes.

References in corpus (2)

• A Survey of Network-based Intrusion Detection Data Sets
• A State-of-the-Art Review on IoT botnet Attack Detection

Cited by in corpus (13)

• E-GraphSAGE: A Graph Neural Network based Intrusion Detection System for IoT
• Towards a Standard Feature Set for Network Intrusion Detection System Datasets
• A Cyber Threat Intelligence Sharing Scheme based on Federated Learning for Network Intrusion Detection
• Evaluating Standard Feature Sets Towards Increased Generalisability and Explainability of ML-based Network Intrusion Detection
• The Cross-evaluation of Machine Learning-based Network Intrusion Detection Systems
• DI-NIDS: Domain Invariant Network Intrusion Detection System
• Towards Explainable Network Intrusion Detection using Large Language Models
• Benchmarking the Benchmark -- Analysis of Synthetic NIDS Datasets
• 3D-IDS: Doubly Disentangled Dynamic Intrusion Detection
• CoAP-DoS: An IoT Network Intrusion Dataset
• Cyber Attacks Prevention Towards Prosumer-based EV Charging Stations: An Edge-assisted Federated Prototype Knowledge Distillation Approach
• A systematic literature review of unsupervised learning algorithms for anomalous traffic detection based on flows
• Intelligent DoS and DDoS Detection: A Hybrid GRU-NTM Approach to Network Security