Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

Abstract

Data Poisoning attacks modify training data to maliciously control a model trained on such data. In this work, we focus on targeted poisoning attacks which cause a reclassification of an unmodified test image and as such breach model integrity. We consider a particularly malicious poisoning attack that is both "from scratch" and "clean label", meaning we analyze an attack that successfully works against new, randomly initialized models, and is nearly imperceptible to humans, all while perturbing only a small fraction of the training data. Previous poisoning attacks against deep neural networks in this setting have been limited in scope and success, working only in simplified settings or being prohibitively expensive for large datasets. The central mechanism of the new attack is matching the gradient direction of malicious examples. We analyze why this works, supplement with practical considerations. and show its threat to real-world practitioners, finding that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset. Finally we demonstrate the limitations of existing defensive strategies against such an attack, concluding that data poisoning is a credible threat, even for large-scale deep learning systems.

arXiv commentFirst two authors contributed equally. Last two authors contributed equally. 21 pages, 11 figures. Published at ICLR 2021

References in corpus (27)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning with Differential Privacy
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Intriguing properties of neural networks
• Understanding Black-box Predictions via Influence Functions
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Poisoning Attacks against Support Vector Machines
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Poisoning Attacks to Graph-Based Recommender Systems
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Adversarial Robustness through Local Linearization
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering
• LaVAN: Localized and Visible Adversarial Noise
• Detection of Adversarial Training Examples in Poisoning Attacks through Anomaly Detection
• Stronger Data Poisoning Attacks Break Data Sanitization Defenses
• MetaPoison: Practical General-purpose Clean-label Data Poisoning
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks
• Adversarial Machine Learning -- Industry Perspectives
• Poisoning Attacks with Generative Adversarial Nets
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Biometric Backdoors: A Poisoning Attack Against Unsupervised Template Updating
• Input Similarity from the Neural Network Perspective
• Deep Neural Network Fingerprinting by Conferrable Adversarial Examples
• A Marauder's Map of Security and Privacy in Machine Learning
• Harmonic Adversarial Attack Method

Cited by in corpus (13)

• Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks
• Adversarial Examples Make Strong Poisons
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations
• Adversarial Sampling for Fairness Testing in Deep Neural Network
• Defending Against Backdoor Attacks in Natural Language Generation
• Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release
• Data Leverage: A Framework for Empowering the Public in its Relationship with Technology Companies
• Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
• Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training
• Model-Targeted Poisoning Attacks with Provable Convergence
• Disrupting Model Training with Adversarial Shortcuts
• TRIGS: Trojan Identification from Gradient-based Signatures
• A Separation Result Between Data-oblivious and Data-aware Poisoning Attacks