Security Evaluation of Pattern Classifiers under Attack

Abstract

Pattern classification systems are commonly used in adversarial applications, like biometric authentication, network intrusion detection, and spam filtering, in which data can be purposely manipulated by humans to undermine their operation. As this adversarial scenario is not taken into account by classical design methods, pattern classification systems may exhibit vulnerabilities, whose exploitation may severely affect their performance, and consequently limit their practical utility. Extending pattern classification theory and design methods to adversarial settings is thus a novel and very relevant research direction, which has not yet been pursued in a systematic way. In this paper, we address one of the main open issues: evaluating at design phase the security of pattern classifiers, namely, the performance degradation under potential attacks they may incur during operation. We propose a framework for empirical evaluation of classifier security that formalizes and generalizes the main ideas proposed in the literature, and give examples of its use in three real applications. Reported results show that security evaluation can provide a more complete understanding of the classifier's behavior in adversarial environments, and lead to better design choices.

References in corpus (1)

• Poisoning Attacks against Support Vector Machines

Cited by in corpus (34)

• Evasion Attacks against Machine Learning at Test Time
• Adversarial Feature Selection against Evasion Attacks
• Adversarial Deep Ensemble: Evasion Attacks and Defenses for Malware Detection
• Modeling Realistic Adversarial Attacks against Network Intrusion Detection Systems
• The Cross-evaluation of Machine Learning-based Network Intrusion Detection Systems
• Revisiting Adversarially Learned Injection Attacks Against Recommender Systems
• Randomized Prediction Games for Adversarial Machine Learning
• Characterizing and evaluating adversarial examples for Offline Handwritten Signature Verification
• Weight Poisoning Attacks on Pre-trained Models
• Towards Adversarial Realism and Robust Learning for IoT Intrusion Detection and Classification
• Statistical Meta-Analysis of Presentation Attacks for Secure Multibiometric Systems
• Can Adversarial Network Attack be Defended?
• A Survey on Resilient Machine Learning
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges
• On Security and Sparsity of Linear Classifiers for Adversarial Settings
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Detecting Adversarial Examples through Nonlinear Dimensionality Reduction
• Lower Bounds for Adversarially Robust PAC Learning
• Intelligent Systems Design for Malware Classification Under Adversarial Conditions
• Advanced Evasion Attacks and Mitigations on Practical ML-Based Phishing Website Classifiers
• Verifying Robustness of Gradient Boosted Models
• Adversarial Transfer Attacks With Unknown Data and Class Overlap
• Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent
• Defending Distributed Classifiers Against Data Poisoning Attacks
• Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications
• Robust Spammer Detection by Nash Reinforcement Learning
• Generating Adversarial Examples with an Optimized Quality
• Can't Boil This Frog: Robustness of Online-Trained Autoencoder-Based Anomaly Detectors to Adversarial Poisoning Attacks
• Defending Regression Learners Against Poisoning Attacks
• Adversarial Attacks for Multi-view Deep Models
• On Intrinsic Dataset Properties for Adversarial Machine Learning
• Adversarial Machine Learning for Cybersecurity and Computer Vision: Current Developments and Challenges
• Certified Robustness of Graph Classification against Topology Attack with Randomized Smoothing
• Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning