Multi-Stage Optimized Machine Learning Framework for Network Intrusion Detection

Abstract

Cyber-security garnered significant attention due to the increased dependency of individuals and organizations on the Internet and their concern about the security and privacy of their online activities. Several previous machine learning (ML)-based network intrusion detection systems (NIDSs) have been developed to protect against malicious online behavior. This paper proposes a novel multi-stage optimized ML-based NIDS framework that reduces computational complexity while maintaining its detection performance. This work studies the impact of oversampling techniques on the models' training sample size and determines the minimal suitable training sample size. Furthermore, it compares between two feature selection techniques, information gain and correlation-based, and explores their effect on detection performance and time complexity. Moreover, different ML hyper-parameter optimization techniques are investigated to enhance the NIDS's performance. The performance of the proposed framework is evaluated using two recent intrusion detection datasets, the CICIDS 2017 and the UNSW-NB 2015 datasets. Experimental results show that the proposed model significantly reduces the required training sample size (up to 74%) and feature set size (up to 50%). Moreover, the model performance is enhanced with hyper-parameter optimization with detection accuracies over 99% for both datasets, outperforming recent literature works by 1-2% higher accuracy and 1-2% lower false alarm rate.

14 Pages, 13 Figures, 4 tables, Published IEEE Transactions on Network and Service Management ( Early Access )

References in corpus (13)

• Scikit-learn: Machine Learning in Python
• SMOTE: Synthetic Minority Over-sampling Technique
• Practical Bayesian Optimization of Machine Learning Algorithms
• On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice
• Feature Selection: A Data Perspective
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Tree-based Intelligent Intrusion Detection System in Internet of Vehicles
• Systematic Ensemble Model Selection Approach for Educational Data Mining
• Bayesian Optimization with Machine Learning Algorithms Towards Anomaly Detection
• Multi-split Optimized Bagging Ensemble Model Selection for Multi-class Educational Data Mining
• Relationship between Student Engagement and Performance in e-Learning Environment Using Association Rules
• DNS Typo-squatting Domain Detection: A Data Analytics & Machine Learning Based Approach
• Data Mining with Big Data in Intrusion Detection Systems: A Systematic Literature Review

Cited by in corpus (10)

• MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehicles
• Supervised Feature Selection Techniques in Network Intrusion Detection: a Critical Review
• Machine Learning Towards Intelligent Systems: Applications, Challenges, and Opportunities
• The Cross-evaluation of Machine Learning-based Network Intrusion Detection Systems
• Bayesian Hyperparameter Optimization for Deep Neural Network-Based Network Intrusion Detection
• IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective
• PWPAE: An Ensemble Framework for Concept Drift Adaptation in IoT Data Streams
• Multi-Perspective Content Delivery Networks Security Framework Using Optimized Unsupervised Anomaly Detection
• A Transfer Learning Framework for Anomaly Detection Using Model of Normality
• Improving the Reliability of Network Intrusion Detection Systems through Dataset Integration