Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning

Abstract

Machine learning is a popular approach to signatureless malware detection because it can generalize to never-before-seen malware families and polymorphic strains. This has resulted in its practical use for either primary detection engines or for supplementary heuristic detection by anti-malware vendors. Recent work in adversarial machine learning has shown that deep learning models are susceptible to gradient-based attacks, whereas non-differentiable models that report a score can be attacked by genetic algorithms that aim to systematically reduce the score. We propose a more general framework based on reinforcement learning (RL) for attacking static portable executable (PE) anti-malware engines. The general framework does not require a differentiable model nor does it require the engine to produce a score. Instead, an RL agent is equipped with a set of functionality-preserving operations that it may perform on the PE file. Through a series of games played against the anti-malware engine, it learns which sequences of operations are likely to result in evading the detector for any given malware sample. This enables completely black-box attacks against static PE anti-malware, and produces functional evasive malware samples as a direct result. We show in experiments that our method can attack a gradient-boosted machine learning model with evasion rates that are substantial and appear to be strongly dependent on the dataset. We demonstrate that attacks against this model appear to also evade components of publicly hosted antivirus engines. Adversarial training results are also presented: by retraining the model on evasive ransomware samples, a subsequent attack is 33% less effective. However, there are overfitting dangers when adversarial training, which we note. We release code to allow researchers to reproduce and improve this approach.

References in corpus (7)

• Playing Atari with Deep Reinforcement Learning
• Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN
• Adversarial Perturbations Against Deep Neural Networks for Malware Classification
• Sample Efficient Actor-Critic with Experience Replay
• Deep Neural Network Based Malware Detection Using Two Dimensional Binary Program Features
• DeepDGA: Adversarially-Tuned Domain Generation and Detection
• Generic Black-Box End-to-End Attack Against State of the Art API Call Based Malware Classifiers

Cited by in corpus (29)

• Adversarial Attacks against Windows PE Malware Detection: A Survey of the State-of-the-Art
• EvadeDroid: A Practical Evasion Attack on Machine Learning for Black-box Android Malware Detection
• I-MAD: Interpretable Malware Detector Using Galaxy Transformer
• Arms Race in Adversarial Malware Detection: A Survey
• Machine Learning (In) Security: A Stream of Problems
• AdvSQLi: Generating Adversarial SQL Injections against Real-world WAF-as-a-service
• Creating Valid Adversarial Examples of Malware
• EvilModel 2.0: Bringing Neural Network Models into Malware Attacks
• MAB-Malware: A Reinforcement Learning Framework for Attacking Static Malware Classifiers
• Enhancing the Insertion of NOP Instructions to Obfuscate Malware via Deep Reinforcement Learning
• Automating Privilege Escalation with Deep Reinforcement Learning
• Query-Free Evasion Attacks Against Machine Learning-Based Malware Detectors with Generative Adversarial Networks
• Binary Black-box Evasion Attacks Against Deep Learning-based Static Malware Detectors with Adversarial Byte-Level Language Model
• DRLDO: A novel DRL based De-ObfuscationSystem for Defense against Metamorphic Malware
• Stealing and Evading Malware Classifiers and Antivirus at Low False Positive Conditions
• A Comparison of Adversarial Learning Techniques for Malware Detection
• Machine Learning for Windows Malware Detection and Classification: Methods, Challenges and Ongoing Research
• The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning
• Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Difference
• Combining Generators of Adversarial Malware Examples to Increase Evasion Rate
• EGAN: Evolutional GAN for Ransomware Evasion
• Black-box Attacks Against Neural Binary Function Detection
• Updating Windows Malware Detectors: Balancing Robustness and Regression against Adversarial EXEmples
• Semantic-preserving Reinforcement Learning Attack Against Graph Neural Networks for Malware Detection
• Evading Malware Classifiers via Monte Carlo Mutant Feature Discovery
• MDEA: Malware Detection with Evolutionary Adversarial Learning
• Blackbox Trojanising of Deep Learning Models : Using non-intrusive network structure and binary alterations
• A Comparison of State-of-the-Art Techniques for Generating Adversarial Malware Binaries
• Deep Detector Health Management under Adversarial Campaigns