Security Evaluation of Pattern Classifiers under Attack

Abstract

Pattern classification systems are commonly used in adversarial applications, like biometric authentication, network intrusion detection, and spam filtering, in which data can be purposely manipulated by humans to undermine their operation. As this adversarial scenario is not taken into account by classical design methods, pattern classification systems may exhibit vulnerabilities, whose exploitation may severely affect their performance, and consequently limit their practical utility. Extending pattern classification theory and design methods to adversarial settings is thus a novel and very relevant research direction, which has not yet been pursued in a systematic way. In this paper, we address one of the main open issues: evaluating at design phase the security of pattern classifiers, namely, the performance degradation under potential attacks they may incur during operation. We propose a framework for empirical evaluation of classifier security that formalizes and generalizes the main ideas proposed in the literature, and give examples of its use in three real applications. Reported results show that security evaluation can provide a more complete understanding of the classifier's behavior in adversarial environments, and lead to better design choices.

References in corpus (2)

• Machine Learning in Automated Text Categorization
• Poisoning Attacks against Support Vector Machines

Cited by in corpus (72)

• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Adversarial Attacks on Neural Networks for Graph Data
• Evasion Attacks against Machine Learning at Test Time
• Certified Defenses against Adversarial Examples
• Is feature selection secure against training data poisoning?
• Adversarial Feature Selection against Evasion Attacks
• Support Vector Machines under Adversarial Label Contamination
• Adversarial Deep Ensemble: Evasion Attacks and Defenses for Malware Detection
• Modeling Realistic Adversarial Attacks against Network Intrusion Detection Systems
• Adversarial EXEmples: A Survey and Experimental Evaluation of Practical Attacks on Machine Learning for Windows Malware Detection
• The Cross-evaluation of Machine Learning-based Network Intrusion Detection Systems
• Survey of Network Intrusion Detection Methods from the Perspective of the Knowledge Discovery in Databases Process
• Towards Adversarial Malware Detection: Lessons Learned from PDF-based Attacks
• Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics
• Revisiting Adversarially Learned Injection Attacks Against Recommender Systems
• Handling Adversarial Concept Drift in Streaming Data
• EvadeDroid: A Practical Evasion Attack on Machine Learning for Black-box Android Malware Detection
• Randomized Prediction Games for Adversarial Machine Learning
• Characterizing and evaluating adversarial examples for Offline Handwritten Signature Verification
• Weight Poisoning Attacks on Pre-trained Models
• Towards Adversarial Realism and Robust Learning for IoT Intrusion Detection and Classification
• Data Driven Exploratory Attacks on Black Box Classifiers in Adversarial Domains
• Fault Sneaking Attack: a Stealthy Framework for Misleading Deep Neural Networks
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• SoK: Realistic Adversarial Attacks and Defenses for Intelligent Network Intrusion Detection
• Arms Race in Adversarial Malware Detection: A Survey
• Adversarial classification: An adversarial risk analysis approach
• Statistical Meta-Analysis of Presentation Attacks for Secure Multibiometric Systems
• Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspective
• Mitigating Adversarial Gray-Box Attacks Against Phishing Detectors
• Are Generative Classifiers More Robust to Adversarial Attacks?
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Dos and Don'ts of Machine Learning in Computer Security
• Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks
• Adversarial Detection of Flash Malware: Limitations and Open Issues
• Adversarial Machine Learning: Bayesian Perspectives
• Can Adversarial Network Attack be Defended?
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing
• Improving Robustness of ML Classifiers against Realizable Evasion Attacks Using Conserved Features
• Non-Negative Networks Against Adversarial Attacks
• A Survey on Resilient Machine Learning
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges
• On Security and Sparsity of Linear Classifiers for Adversarial Settings
• Detecting Adversarial Examples through Nonlinear Dimensionality Reduction
• Adversarially Regularized Graph Attention Networks for Inductive Learning on Partially Labeled Graphs
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• A Dynamic-Adversarial Mining Approach to the Security of Machine Learning
• Lower Bounds for Adversarially Robust PAC Learning
• Intelligent Systems Design for Malware Classification Under Adversarial Conditions
• Verifying Robustness of Gradient Boosted Models
• Advanced Evasion Attacks and Mitigations on Practical ML-Based Phishing Website Classifiers
• Security Theater: On the Vulnerability of Classifiers to Exploratory Attacks
• Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples
• Adversarial Transfer Attacks With Unknown Data and Class Overlap
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Poisoning Behavioral Malware Clustering
• Enhancing the Antidote: Improved Pointwise Certifications against Poisoning Attacks
• Defending Distributed Classifiers Against Data Poisoning Attacks
• Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications
• Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent
• Robust Spammer Detection by Nash Reinforcement Learning
• Defending Regression Learners Against Poisoning Attacks
• Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design
• SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation
• Generating Adversarial Examples with an Optimized Quality
• Can't Boil This Frog: Robustness of Online-Trained Autoencoder-Based Anomaly Detectors to Adversarial Poisoning Attacks
• Certified Robustness of Graph Classification against Topology Attack with Randomized Smoothing
• Adversarial Machine Learning for Cybersecurity and Computer Vision: Current Developments and Challenges
• Adversarial Attacks for Multi-view Deep Models
• On Intrinsic Dataset Properties for Adversarial Machine Learning
• We Can Always Catch You: Detecting Adversarial Patched Objects WITH or WITHOUT Signature
• Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning