Low-Quality Training Data Only? A Robust Framework for Detecting Encrypted Malicious Network Traffic

Abstract

Machine learning (ML) is promising in accurately detecting malicious flows in encrypted network traffic; however, it is challenging to collect a training dataset that contains a sufficient amount of encrypted malicious data with correct labels. When ML models are trained with low-quality training data, they suffer degraded performance. In this paper, we aim at addressing a real-world low-quality training dataset problem, namely, detecting encrypted malicious traffic generated by continuously evolving malware. We develop RAPIER that fully utilizes different distributions of normal and malicious traffic data in the feature space, where normal data is tightly distributed in a certain area and the malicious data is scattered over the entire feature space to augment training data for model training. RAPIER includes two pre-processing modules to convert traffic into feature vectors and correct label noises. We evaluate our system on two public datasets and one combined dataset. With 1000 samples and 45% noises from each dataset, our system achieves the F1 scores of 0.770, 0.776, and 0.855, respectively, achieving average improvements of 352.6%, 284.3%, and 214.9% over the existing methods, respectively. Furthermore, We evaluate RAPIER with a real-world dataset obtained from a security enterprise. RAPIER effectively achieves encrypted malicious traffic detection with the best F1 score of 0.773 and improves the F1 score of existing methods by an average of 272.5%.

References in corpus (25)

• SMOTE: Synthetic Minority Over-sampling Technique
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Semi-Supervised Learning with Deep Generative Models
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Generalizing from a Few Examples: A Survey on Few-Shot Learning
• ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification
• Training Convolutional Networks with Noisy Labels
• Efficient GAN-Based Anomaly Detection
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• MADE: Masked Autoencoder for Distribution Estimation
• How does Disagreement Help Generalization against Label Corruption?
• Robust Loss Functions under Label Noise for Deep Neural Networks
• Part-dependent Label Noise: Towards Instance-dependent Label Noise
• Symmetric Cross Entropy for Robust Learning with Noisy Labels
• Decoupling "when to update" from "how to update"
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Curriculum Loss: Robust Learning and Generalization against Label Corruption
• Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
• AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine Learning
• On the Computational Efficiency of Training Neural Networks
• TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time
• Deep Fingerprinting: Undermining Website Fingerprinting Defenses with Deep Learning
• Data Augmentation Can Improve Robustness
• Detecting Unknown Encrypted Malicious Traffic in Real Time via Flow Interaction Graph Analysis