Manipulating SGD with Data Ordering Attacks

Abstract

Machine learning is vulnerable to a wide variety of attacks. It is now well understood that by changing the underlying data distribution, an adversary can poison the model trained with it or introduce backdoors. In this paper we present a novel class of training-time attacks that require no changes to the underlying dataset or model architecture, but instead only change the order in which data are supplied to the model. In particular, we find that the attacker can either prevent the model from learning, or poison it to learn behaviours specified by the attacker. Furthermore, we find that even a single adversarially-ordered epoch can be enough to slow down model learning, or even to reset all of the learning progress. Indeed, the attacks presented here are not specific to the model or dataset, but rather target the stochastic nature of modern learning procedures. We extensively evaluate our attacks on computer vision and natural language benchmarks to find that the adversary can disrupt model training and even introduce backdoors.

References in corpus (13)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Language Models are Few-Shot Learners
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• DARTS: Differentiable Architecture Search
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Revisiting Small Batch Training for Deep Neural Networks
• Dynamic Channel Pruning: Feature Boosting and Suppression
• Batch Normalization is a Cause of Adversarial Vulnerability
• How to Start Training: The Effect of Initialization and Architecture
• Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?
• Poisoned classifiers are not only backdoored, they are fundamentally broken

Cited by in corpus (5)

• Randomness In Neural Network Training: Characterizing The Impact of Tooling
• SoK: Machine Learning Governance
• Disrupting Model Training with Adversarial Shortcuts
• Rethinking Image-Scaling Attacks: The Interplay Between Vulnerabilities in Machine Learning Systems
• Accumulative Poisoning Attacks on Real-time Data