On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping

Abstract

Machine learning algorithms are vulnerable to data poisoning attacks. Prior taxonomies that focus on specific scenarios, e.g., indiscriminate or targeted, have enabled defenses for the corresponding subset of known attacks. Yet, this introduces an inevitable arms race between adversaries and defenders. In this work, we study the feasibility of an attack-agnostic defense relying on artifacts that are common to all poisoning attacks. Specifically, we focus on a common element between all attacks: they modify gradients computed to train the model. We identify two main artifacts of gradients computed in the presence of poison: (1) their ${\ell }_2$ norms have significantly higher magnitudes than those of clean gradients, and (2) their orientation differs from clean gradients. Based on these observations, we propose the prerequisite for a generic poisoning defense: it must bound gradient magnitudes and minimize differences in orientation. We call this gradient shaping. As an exemplar tool to evaluate the feasibility of gradient shaping, we use differentially private stochastic gradient descent (DP-SGD), which clips and perturbs individual gradients during training to obtain privacy guarantees. We find that DP-SGD, even in configurations that do not result in meaningful privacy guarantees, increases the model's robustness to indiscriminate attacks. It also mitigates worst-case targeted attacks and increases the adversary's cost in multi-poison scenarios. The only attack we find DP-SGD to be ineffective against is a strong, yet unrealistic, indiscriminate attack. Our results suggest that, while we currently lack a generic poisoning defense, gradient shaping is a promising direction for future research.

References in corpus (14)

• Deep Learning with Differential Privacy
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• How To Backdoor Federated Learning
• Poisoning Attacks against Support Vector Machines
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Is feature selection secure against training data poisoning?
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Qualitatively characterizing neural network optimization problems
• Sever: A Robust Meta-Algorithm for Stochastic Optimization
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• Data Poisoning against Differentially-Private Learners: Attacks and Defenses

Cited by in corpus (23)

• Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
• Blind Backdoors in Deep Learning Models
• Rethinking the Trigger of Backdoor Attack
• Auditing Differentially Private Machine Learning: How Private is Private SGD?
• You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
• Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching
• Machine Learning Security against Data Poisoning: Are We There Yet?
• Adversarial Examples Make Strong Poisons
• On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approach
• Poisoning the Unlabeled Dataset of Semi-Supervised Learning
• Handcrafted Backdoors in Deep Neural Networks
• HaS-Nets: A Heal and Select Mechanism to Defend DNNs Against Backdoor Attacks for Data Collection Scenarios
• SoK: Machine Learning Governance
• Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release
• Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
• FaceHack: Triggering backdoored facial recognition systems using facial characteristics
• Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training
• Model-Targeted Poisoning Attacks with Provable Convergence
• Backdoor Learning Curves: Explaining Backdoor Poisoning Beyond Influence Functions
• A General Framework for Defending Against Backdoor Attacks via Influence Graph
• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
• Enhancing the Antidote: Improved Pointwise Certifications against Poisoning Attacks
• A BIC-based Mixture Model Defense against Data Poisoning Attacks on Classifiers