Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning

Abstract

The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to compromise the model's performance at test time. Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical review of the field is still missing. In this survey, we provide a comprehensive systematization of poisoning attacks and defenses in machine learning, reviewing more than 100 papers published in the field in the last 15 years. We start by categorizing the current threat models and attacks, and then organize existing defenses accordingly. While we focus mostly on computer-vision applications, we argue that our systematization also encompasses state-of-the-art attacks and defenses for other data modalities. Finally, we discuss existing resources for research in poisoning, and shed light on the current limitations and open research questions in this research field.

35 pages, Accepted at ACM Computing Surveys

References in corpus (22)

• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Evasion Attacks against Machine Learning at Test Time
• On Evaluating Adversarial Robustness
• Can You Really Backdoor Federated Learning?
• Support Vector Machines under Adversarial Label Contamination
• Generative Poisoning Attack Method Against Neural Networks
• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations
• Machine Learning Security in Industry: A Quantitative Survey
• Anti-Backdoor Learning: Training Clean Models on Poisoned Data
• Adversarial Examples Make Strong Poisons
• ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations
• CLEANN: Accelerated Trojan Shield for Embedded Neural Networks
• Planting Undetectable Backdoors in Machine Learning Models
• Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers
• Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release
• TAD: Trigger Approximation based Black-box Trojan Detection for AI
• Baseline Pruning-Based Approach to Trojan Detection in Neural Networks
• Regularization Can Help Mitigate Poisoning Attacks... with the Right Hyperparameters

Cited by in corpus (10)

• Adversarial attacks and defenses in explainable artificial intelligence: A survey
• LLM-Assisted Crisis Management: Building Advanced LLM Platforms for Effective Emergency Response and Public Collaboration
• Machine Learning Security in Industry: A Quantitative Survey
• Machine Learning Security against Data Poisoning: Are We There Yet?
• On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approach
• A Backdoor Approach with Inverted Labels Using Dirty Label-Flipping Attacks
• Securing Contrastive mmWave-based Human Activity Recognition against Adversarial Label Flipping
• Can Artificial Intelligence solve the blockchain oracle problem? Unpacking the Challenges and Possibilities
• Dye4AI: Assuring Data Boundary on Generative AI Services
• Reviewing Model Collapse and Countermeasures