Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey

Abstract

With significant advancements in Transformers LLMs, NLP has extended its reach into many research fields due to its enhanced capabilities in text generation and user interaction. One field benefiting greatly from these advancements is cybersecurity. In cybersecurity, many parameters that need to be protected and exchanged between senders and receivers are in the form of text and tabular data, making NLP a valuable tool in enhancing the security measures of communication protocols. This survey paper provides a comprehensive analysis of the utilization of Transformers and LLMs in cyber-threat detection systems. The methodology of paper selection and bibliometric analysis is outlined to establish a rigorous framework for evaluating existing research. The fundamentals of Transformers are discussed, including background information on various cyber-attacks and datasets commonly used in this field. The survey explores the application of Transformers in IDSs, focusing on different architectures such as Attention-based models, LLMs like BERT and GPT, CNN/LSTM-Transformer hybrids, emerging approaches like ViTs, among others. Furthermore, it explores the diverse environments and applications where Transformers and LLMs-based IDS have been implemented, including computer networks, IoT devices, critical infrastructure protection, cloud computing, SDN, as well as in autonomous vehicles. The paper also addresses research challenges and future directions in this area, identifying key issues such as interpretability, scalability, and adaptability to evolving threats, and more. Finally, the conclusion summarizes the findings and highlights the significance of Transformers and LLMs in enhancing cyber-threat detection capabilities, while also outlining potential avenues for further research and development.

arXiv commentarXiv admin note: text overlap with arXiv:2405.04760 by other authors

References in corpus (36)

• Knowledge Distillation: A Survey
• Longformer: The Long-Document Transformer
• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
• To prune, or not to prune: exploring the efficacy of pruning for model compression
• The Rise and Potential of Large Language Model Based Agents: A Survey
• FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systems
• Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey
• Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization
• Deep transfer learning for intrusion detection in industrial control networks: A comprehensive review
• Deep Learning for Steganalysis of Diverse Data Types: A review of methods, taxonomy, challenges and future directions
• TinyLlama: An Open-Source Small Language Model
• On Generalisability of Machine Learning-based Network Intrusion Detection Systems
• Towards Explainable Network Intrusion Detection using Large Language Models
• PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
• From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloads
• Large Language Models for Cyber Security: A Systematic Literature Review
• BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT
• Flow-based Network Intrusion Detection Based on BERT Masked Language Model
• LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning
• From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?
• A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law
• LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks
• Time for aCTIon: Automated Analysis of Cyber Threat Intelligence in the Wild
• LLMs Killed the Script Kiddie: How Agents Supported by Large Language Models Change the Landscape of Network Threat Testing
• Mapping LLM Security Landscapes: A Comprehensive Stakeholder Risk Assessment Proposal
• Critical Infrastructure Protection: Generative AI, Challenges, and Opportunities
• On the Uses of Large Language Models to Interpret Ambiguous Cyberattack Descriptions
• TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models
• Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks
• CVE-driven Attack Technique Prediction with Semantic Information Extraction and a Domain-specific Language Model
• Automated CVE Analysis for Threat Prioritization and Impact Prediction
• Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities
• PMANet: Malicious URL detection via post-trained language model guided multi-level feature attention network
• DoLLM: How Large Language Models Understanding Network Flow Data to Detect Carpet Bombing DDoS
• AnyTaskTune: Advanced Domain-Specific Solutions through Task-Fine-Tuning
• Depending on yourself when you should: Mentoring LLM with RL agents to become the master in cybersecurity games

Cited by in corpus (6)

• A Robust Cross-Domain IDS using BiGRU-LSTM-Attention for Medical and Industrial IoT Security
• Advances in Artificial Intelligence: A Review for the Creative Industries
• LLM-Driven APT Detection for 6G Wireless Networks: A Systematic Review and Taxonomy
• Machine Learning and Transformers for Thyroid Carcinoma Diagnosis: A Review
• Transfer Learning-Based Deep Residual Learning for Speech Recognition in Clean and Noisy Environments
• Hybrid ResNet-1D-BiGRU with Multi-Head Attention for Cyberattack Detection in Industrial IoT Environments