Accumulative Poisoning Attacks on Real-time Data

Abstract

Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy. When trained on offline datasets, poisoning adversaries have to inject the poisoned data in advance before training, and the order of feeding these poisoned batches into the model is stochastic. In contrast, practical systems are more usually trained/fine-tuned on sequentially captured real-time data, in which case poisoning adversaries could dynamically poison each data batch according to the current model state. In this paper, we focus on the real-time settings and propose a new attacking strategy, which affiliates an accumulative phase with poisoning attacks to secretly (i.e., without affecting accuracy) magnify the destructive effect of a (poisoned) trigger batch. By mimicking online learning and federated learning on MNIST and CIFAR-10, we show that model accuracy significantly drops by a single update step on the trigger batch after the accumulative phase. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects, with no need to explore complex techniques.

NeurIPS 2021

References in corpus (16)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Poisoning Attacks against Support Vector Machines
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• Can Adversarial Weight Perturbations Inject Neural Backdoors?
• Label-Consistent Backdoor Attacks
• Data Poisoning Attacks on Stochastic Bandits
• Benchmarking Adversarial Robustness
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations
• HaS-Nets: A Heal and Select Mechanism to Defend DNNs Against Backdoor Attacks for Data Collection Scenarios
• TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation
• A Backdoor Attack against 3D Point Cloud Classifiers
• Regularization Can Help Mitigate Poisoning Attacks... with the Right Hyperparameters
• Black-box Detection of Backdoor Attacks with Limited Information and Data