Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning

Abstract

The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to compromise the model's performance at test time. Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical review of the field is still missing. In this survey, we provide a comprehensive systematization of poisoning attacks and defenses in machine learning, reviewing more than 100 papers published in the field in the last 15 years. We start by categorizing the current threat models and attacks, and then organize existing defenses accordingly. While we focus mostly on computer-vision applications, we argue that our systematization also encompasses state-of-the-art attacks and defenses for other data modalities. Finally, we discuss existing resources for research in poisoning, and shed light on the current limitations and open research questions in this research field.

35 pages, Accepted at ACM Computing Surveys

References in corpus (37)

• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Evasion Attacks against Machine Learning at Test Time
• On Evaluating Adversarial Robustness
• Can You Really Backdoor Federated Learning?
• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
• Support Vector Machines under Adversarial Label Contamination
• Generative Poisoning Attack Method Against Neural Networks
• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Rethinking the Trigger of Backdoor Attack
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations
• Backdoor Scanning for Deep Neural Networks through K-Arm Optimization
• RAB: Provable Robustness Against Backdoor Attacks
• Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization
• A black-box adversarial attack for poisoning clustering
• Machine Learning Security in Industry: A Quantitative Survey
• Advbox: a toolbox to generate adversarial examples that fool neural networks
• Backdoor Learning: A Survey
• Defending against Backdoor Attack on Deep Neural Networks
• Anti-Backdoor Learning: Training Clean Models on Poisoned Data
• Adversarial Examples Make Strong Poisons
• DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
• ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations
• CLEANN: Accelerated Trojan Shield for Embedded Neural Networks
• Planting Undetectable Backdoors in Machine Learning Models
• Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers
• Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release
• TAD: Trigger Approximation based Black-box Trojan Detection for AI
• Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
• Defense Against Reward Poisoning Attacks in Reinforcement Learning
• Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples
• Regularization Can Help Mitigate Poisoning Attacks... with the Right Hyperparameters
• Baseline Pruning-Based Approach to Trojan Detection in Neural Networks
• Backdoor Learning Curves: Explaining Backdoor Poisoning Beyond Influence Functions

Cited by in corpus (11)

• Adversarial attacks and defenses in explainable artificial intelligence: A survey
• LLM-Assisted Crisis Management: Building Advanced LLM Platforms for Effective Emergency Response and Public Collaboration
• Machine Learning Security in Industry: A Quantitative Survey
• Machine Learning Security against Data Poisoning: Are We There Yet?
• On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approach
• A Backdoor Approach with Inverted Labels Using Dirty Label-Flipping Attacks
• Securing Contrastive mmWave-based Human Activity Recognition against Adversarial Label Flipping
• Can Artificial Intelligence solve the blockchain oracle problem? Unpacking the Challenges and Possibilities
• Backdoor Learning Curves: Explaining Backdoor Poisoning Beyond Influence Functions
• Dye4AI: Assuring Data Boundary on Generative AI Services
• Reviewing Model Collapse and Countermeasures