ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification

Abstract

Encrypted traffic classification requires discriminative and robust traffic representation captured from content-invisible and imbalanced traffic data for accurate classification, which is challenging but indispensable to achieve network security and network management. The major limitation of existing solutions is that they highly rely on the deep features, which are overly dependent on data size and hard to generalize on unseen data. How to leverage the open-domain unlabeled traffic data to learn representation with strong generalization ability remains a key challenge. In this paper,we propose a new traffic representation model called Encrypted Traffic Bidirectional Encoder Representations from Transformer (ET-BERT), which pre-trains deep contextualized datagram-level representation from large-scale unlabeled data. The pre-trained model can be fine-tuned on a small number of task-specific labeled data and achieves state-of-the-art performance across five encrypted traffic classification tasks, remarkably pushing the F1 of ISCX-Tor to 99.2% (4.4% absolute improvement), ISCX-VPN-Service to 98.9% (5.2% absolute improvement), Cross-Platform (Android) to 92.5% (5.4% absolute improvement), CSTNET-TLS 1.3 to 97.4% (10.0% absolute improvement). Notably, we provide explanation of the empirically powerful pre-training model by analyzing the randomness of ciphers. It gives us insights in understanding the boundary of classification ability over encrypted traffic. The code is available at: https://github.com/linwhitehat/ET-BERT.

arXiv commentThis work has been accepted in Security, Privacy, and Trust track at The Web Conference 2022 (WWW'22)(see https://www2022.thewebconf.org/cfp/research/security/)

References in corpus (4)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Deep Learning for Encrypted Traffic Classification: An Overview

Cited by in corpus (10)

• NetLLM: Adapting Large Language Models for Networking
• Low-Quality Training Data Only? A Robust Framework for Detecting Encrypted Malicious Network Traffic
• SoK: Decoding the Enigma of Encrypted Network Traffic Classifiers
• AutoML4ETC: Automated Neural Architecture Search for Real-World Encrypted Traffic Classification
• Mapping the Landscape of Generative AI in Network Monitoring and Management
• The Sweet Danger of Sugar: Debunking Representation Learning for Encrypted Traffic Classification
• Respond to Change with Constancy: Instruction-tuning with LLM for Non-I.I.D. Network Traffic Classification
• Universal Embedding Function for Traffic Classification via QUIC Domain Recognition Pretraining: A Transfer Learning Success
• DecETT: Accurate App Fingerprinting Under Encrypted Tunnels via Dual Decouple-based Semantic Enhancement
• Generalized Encrypted Traffic Classification Using Inter-Flow Signals