Certified Defenses for Data Poisoning Attacks

Abstract

Machine learning systems trained on user-provided data are susceptible to data poisoning attacks, whereby malicious users inject false training data with the aim of corrupting the learned model. While recent work has proposed a number of attacks and defenses, little is understood about the worst-case loss of a defense in the face of a determined attacker. We address this by constructing approximate upper bounds on the loss across a broad family of attacks, for defenders that first perform outlier removal followed by empirical risk minimization. Our approximation relies on two assumptions: (1) that the dataset is large enough for statistical concentration between train and test error to hold, and (2) that outliers within the clean (non-poisoned) data do not have a strong effect on the model. Our bound comes paired with a candidate attack that often nearly matches the upper bound, giving us a powerful tool for quickly assessing defenses on a given dataset. Empirically, we find that even under a simple defense, the MNIST-1-7 and Dogfish datasets are resilient to attack, while in contrast the IMDB sentiment dataset can be driven from 12% to 23% test error by adding only 3% poisoned data.

Appeared at NIPS 2017

References in corpus (7)

• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Stealing Machine Learning Models via Prediction APIs
• Support Vector Machines under Adversarial Label Contamination
• Generative Poisoning Attack Method Against Neural Networks
• Tactics of Adversarial Attack on Deep Reinforcement Learning Agents
• The Statistics of Streaming Sparse Regression
• Poisoning Behavioral Malware Clustering

Cited by in corpus (10)

• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Subnet Replacement: Deployment-stage backdoor attack against deep neural networks in gray-box setting
• Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defence
• Reinforcement Learning for Autonomous Defence in Software-Defined Networking
• Robust Generalization of Quadratic Neural Networks via Function Identification
• Are You Tampering With My Data?
• Formal Verification of Neural Network Controlled Autonomous Systems