Evasion Attacks against Machine Learning at Test Time

Abstract

In security-sensitive applications, the success of machine learning depends on a thorough vetting of their resistance to adversarial data. In one pertinent, well-motivated attack scenario, an adversary may attempt to evade a deployed system at test time by carefully manipulating attack samples. In this work, we present a simple but effective gradient-based approach that can be exploited to systematically assess the security of several, widely-used classification algorithms against evasion attacks. Following a recently proposed framework for security evaluation, we simulate attack scenarios that exhibit different risk levels for the classifier by increasing the attacker's knowledge of the system and her ability to manipulate attack samples. This gives the classifier designer a better picture of the classifier performance under evasion attacks, and allows him to perform a more informed model selection (or parameter setting). We evaluate our approach on the relevant security task of malware detection in PDF files, and show that such systems can be easily evaded. We also sketch some countermeasures suggested by our analysis.

In this paper, in 2013, we were the first to introduce the notion of evasion attacks (adversarial examples) created with high confidence (instead of minimum-distance misclassifications), and the notion of surrogate learners (substitute models). These two concepts are now widely re-used in developing attacks against deep networks (even if not always referring to the ideas reported in this work). arXiv admin note: text overlap with arXiv:1401.7727

References in corpus (1)

• Security Evaluation of Pattern Classifiers under Attack

Cited by in corpus (91)

• Adversarial Attack and Defense on Graph Data: A Survey
• Support Vector Machines under Adversarial Label Contamination
• A General Framework for Adversarial Examples with Objectives
• Detecting Adversarial Image Examples in Deep Networks with Adaptive Noise Reduction
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Privacy Risks of Securing Machine Learning Models against Adversarial Examples
• Adversarial attacks and defenses in explainable artificial intelligence: A survey
• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
• Adversarial EXEmples: A Survey and Experimental Evaluation of Practical Attacks on Machine Learning for Windows Malware Detection
• Survey of Network Intrusion Detection Methods from the Perspective of the Knowledge Discovery in Databases Process
• A Comprehensive Analysis of the Role of Artificial Intelligence and Machine Learning in Modern Digital Forensics and Incident Response
• Machine Learning Based Cyber Attacks Targeting on Controlled Information: A Survey
• Machine Learning for Security in Vehicular Networks: A Comprehensive Survey
• Assessing and Remedying Coverage for a Given Dataset
• Adversarial Attacks Against Deep Generative Models on Data: A Survey
• Towards Adversarial Malware Detection: Lessons Learned from PDF-based Attacks
• Secure and Trustworthy Artificial Intelligence-Extended Reality (AI-XR) for Metaverses
• Robust Android Malware Detection System against Adversarial Attacks using Q-Learning
• Towards quantum enhanced adversarial robustness in machine learning
• Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics
• Attacking Recommender Systems with Augmented User Profiles
• Semantic Robustness of Models of Source Code
• Randomized Prediction Games for Adversarial Machine Learning
• Malware Makeover: Breaking ML-based Static Analysis by Modifying Executable Bytes
• REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data
• Constrained Concealment Attacks against Reconstruction-based Anomaly Detectors in Industrial Control Systems
• Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fitting
• Shilling Black-box Recommender Systems by Learning to Generate Fake User Profiles
• A Framework for Enhancing Deep Neural Networks Against Adversarial Malware
• Procedural Noise Adversarial Examples for Black-Box Attacks on Deep Convolutional Networks
• A Survey of Game Theoretic Approaches for Adversarial Machine Learning in Cybersecurity Tasks
• WAF-A-MoLE: Evading Web Application Firewalls through Adversarial Machine Learning
• Applications of Machine Learning in Cryptography: A Survey
• Adversarial Patch Attacks on Monocular Depth Estimation Networks
• Arms Race in Adversarial Malware Detection: A Survey
• PAD: Towards Principled Adversarial Malware Detection Against Evasion Attacks
• On the Adversarial Robustness of LASSO Based Feature Selection
• Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysis
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis
• Drastic Circuit Depth Reductions with Preserved Adversarial Robustness by Approximate Encoding for Quantum Machine Learning
• On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approach
• Morphence: Moving Target Defense Against Adversarial Examples
• Adversarial Attacks and Defenses in Fault Detection and Diagnosis: A Comprehensive Benchmark on the Tennessee Eastman Process
• Wild Networks: Exposure of 5G Network Infrastructures to Adversarial Examples
• Query-Free Evasion Attacks Against Machine Learning-Based Malware Detectors with Generative Adversarial Networks
• Turning Federated Learning Systems Into Covert Channels
• Generating Black-Box Adversarial Examples for Text Classifiers Using a Deep Reinforced Model
• Adversarial Defense by Latent Style Transformations
• Android HIV: A Study of Repackaging Malware for Evading Machine-Learning Detection
• Adversarial Attack for Uncertainty Estimation: Identifying Critical Regions in Neural Networks
• Two Souls in an Adversarial Image: Towards Universal Adversarial Example Detection using Multi-view Inconsistency
• CARLA-GeAR: a Dataset Generator for a Systematic Evaluation of Adversarial Robustness of Vision Models
• Verifiable Obstacle Detection
• Noise Sensitivity-Based Energy Efficient and Robust Adversary Detection in Neural Networks
• Universal Adversarial Robustness of Texture and Shape-Biased Models
• On Security and Sparsity of Linear Classifiers for Adversarial Settings
• A Robust Defense against Adversarial Attacks on Deep Learning-based Malware Detectors via (De)Randomized Smoothing
• FooBaR: Fault Fooling Backdoor Attack on Neural Network Training
• Bi-Level Poisoning Attack Model and Countermeasure for Appliance Consumption Data of Smart Homes
• ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks
• A Dynamic-Adversarial Mining Approach to the Security of Machine Learning
• Jacobian Regularization for Mitigating Universal Adversarial Perturbations
• Dense Hopfield Networks in the Teacher-Student Setting
• On the Detection of Adaptive Adversarial Attacks in Speaker Verification Systems
• Adversarial Robustness Guarantees for Quantum Classifiers
• Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attacks
• Towards Resistant Audio Adversarial Examples
• A Systematic Evaluation of Adversarial Attacks against Speech Emotion Recognition Models
• Towards a Practical Defense against Adversarial Attacks on Deep Learning-based Malware Detectors via Randomized Smoothing
• Security Theater: On the Vulnerability of Classifiers to Exploratory Attacks
• Stochastic Substitute Training: A Gray-box Approach to Craft Adversarial Examples Against Gradient Obfuscation Defenses
• Black-box Attacks Against Neural Binary Function Detection
• On the Minimal Adversarial Perturbation for Deep Neural Networks with Provable Estimation Error
• HO-FMN: Hyperparameter Optimization for Fast Minimum-Norm Attacks
• Careful What You Wish For: on the Extraction of Adversarially Trained Models
• Making targeted black-box evasion attacks effective and efficient
• Resilient Linear Classification: An Approach to Deal with Attacks on Training Data
• Semantically Adversarial Learnable Filters
• Random Walks for Adversarial Meshes
• When AI reviews science: Can we trust the referee?
• The Ephemeral Threat: Assessing the Security of Algorithmic Trading Systems powered by Deep Learning
• A direct proof of a unified law of robustness for Bregman divergence losses
• Adversarial Robustness on Image Classification with $k$-means
• Defending Network Intrusion Detection Systems Based on Graph Neural Networks Against Structural Adversarial Attacks
• Messing Up 3D Virtual Environments: Transferable Adversarial 3D Objects
• Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier
• Should All Noises Be Treated Equally: Impact of Input Noise Variability on Neural Network Robustness
• On $L^{\infty }$ stability for wave propagation and for linear inverse problems
• When Vision Fails: Text Attacks Against ViT and OCR
• Where Does the Robustness Come from? A Study of the Transformation-based Ensemble Defence