Low-Quality Training Data Only? A Robust Framework for Detecting Encrypted Malicious Network Traffic

Abstract

Machine learning (ML) is promising in accurately detecting malicious flows in encrypted network traffic; however, it is challenging to collect a training dataset that contains a sufficient amount of encrypted malicious data with correct labels. When ML models are trained with low-quality training data, they suffer degraded performance. In this paper, we aim at addressing a real-world low-quality training dataset problem, namely, detecting encrypted malicious traffic generated by continuously evolving malware. We develop RAPIER that fully utilizes different distributions of normal and malicious traffic data in the feature space, where normal data is tightly distributed in a certain area and the malicious data is scattered over the entire feature space to augment training data for model training. RAPIER includes two pre-processing modules to convert traffic into feature vectors and correct label noises. We evaluate our system on two public datasets and one combined dataset. With 1000 samples and 45% noises from each dataset, our system achieves the F1 scores of 0.770, 0.776, and 0.855, respectively, achieving average improvements of 352.6%, 284.3%, and 214.9% over the existing methods, respectively. Furthermore, We evaluate RAPIER with a real-world dataset obtained from a security enterprise. RAPIER effectively achieves encrypted malicious traffic detection with the best F1 score of 0.773 and improves the F1 score of existing methods by an average of 272.5%.

References in corpus (11)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Semi-Supervised Learning with Deep Generative Models
• ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification
• Training Convolutional Networks with Noisy Labels
• MADE: Masked Autoencoder for Distribution Estimation
• How does Disagreement Help Generalization against Label Corruption?
• Robust Loss Functions under Label Noise for Deep Neural Networks
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• On the Computational Efficiency of Training Neural Networks
• Data Augmentation Can Improve Robustness
• Detecting Unknown Encrypted Malicious Traffic in Real Time via Flow Interaction Graph Analysis