The Cross-evaluation of Machine Learning-based Network Intrusion Detection Systems

Abstract

Enhancing Network Intrusion Detection Systems (NIDS) with supervised Machine Learning (ML) is tough. ML-NIDS must be trained and evaluated, operations requiring data where benign and malicious samples are clearly labelled. Such labels demand costly expert knowledge, resulting in a lack of real deployments, as well as on papers always relying on the same outdated data. The situation improved recently, as some efforts disclosed their labelled datasets. However, most past works used such datasets just as a 'yet another' testbed, overlooking the added potential provided by such availability. In contrast, we promote using such existing labelled data to cross-evaluate ML-NIDS. Such approach received only limited attention and, due to its complexity, requires a dedicated treatment. We hence propose the first cross-evaluation model. Our model highlights the broader range of realistic use-cases that can be assessed via cross-evaluations, allowing the discovery of still unknown qualities of state-of-the-art ML-NIDS. For instance, their detection surface can be extended--at no additional labelling cost. However, conducting such cross-evaluations is challenging. Hence, we propose the first framework, XeNIDS, for reliable cross-evaluations based on Network Flows. By using XeNIDS on six well-known datasets, we demonstrate the concealed potential, but also the risks, of cross-evaluations of ML-NIDS.

References in corpus (10)

• A Survey of Network-based Intrusion Detection Data Sets
• Security Evaluation of Pattern Classifiers under Attack
• NetFlow Datasets for Machine Learning-based Network Intrusion Detection Systems
• LUCID: A Practical, Lightweight Deep Learning Solution for DDoS Attack Detection
• Multi-Stage Optimized Machine Learning Framework for Network Intrusion Detection
• Benchmarking datasets for Anomaly-based Network Intrusion Detection: KDD CUP 99 alternatives
• Modeling Realistic Adversarial Attacks against Network Intrusion Detection Systems
• A new method for flow-based network intrusion detection using the inverse Potts model
• A Framework for Cluster and Classifier Evaluation in the Absence of Reference Labels
• Walling up Backdoors in Intrusion Detection Systems

Cited by in corpus (6)

• The Role of Machine Learning in Cybersecurity
• Evaluating Standard Feature Sets Towards Increased Generalisability and Explainability of ML-based Network Intrusion Detection
• Generalizing intrusion detection for heterogeneous networks: A stacked-unsupervised federated learning approach
• On Generalisability of Machine Learning-based Network Intrusion Detection Systems
• SoK: Decoding the Enigma of Encrypted Network Traffic Classifiers
• A Novel Open Set Energy-based Flow Classifier for Network Intrusion Detection