Accumulative Poisoning Attacks on Real-time Data

Abstract

Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy. When trained on offline datasets, poisoning adversaries have to inject the poisoned data in advance before training, and the order of feeding these poisoned batches into the model is stochastic. In contrast, practical systems are more usually trained/fine-tuned on sequentially captured real-time data, in which case poisoning adversaries could dynamically poison each data batch according to the current model state. In this paper, we focus on the real-time settings and propose a new attacking strategy, which affiliates an accumulative phase with poisoning attacks to secretly (i.e., without affecting accuracy) magnify the destructive effect of a (poisoned) trigger batch. By mimicking online learning and federated learning on MNIST and CIFAR-10, we show that model accuracy significantly drops by a single update step on the trigger batch after the accumulative phase. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects, with no need to explore complex techniques.

arXiv commentNeurIPS 2021

References in corpus (37)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Understanding Black-box Predictions via Influence Functions
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Poisoning Attacks against Support Vector Machines
• Detecting Adversarial Samples from Artifacts
• Is feature selection secure against training data poisoning?
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• Detection of Adversarial Training Examples in Poisoning Attacks through Anomaly Detection
• Stronger Data Poisoning Attacks Break Data Sanitization Defenses
• Can Adversarial Weight Perturbations Inject Neural Backdoors?
• First-order Adversarial Vulnerability of Neural Networks and Input Dimension
• Data Poisoning Attacks against Online Learning
• Adversarial Attacks on Stochastic Bandits
• Label-Consistent Backdoor Attacks
• A black-box adversarial attack for poisoning clustering
• Data Poisoning Attacks on Stochastic Bandits
• Benchmarking Adversarial Robustness
• BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Models
• Don't Trigger Me! A Triggerless Backdoor Attack Against Deep Neural Networks
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations
• Explainability-based Backdoor Attacks Against Graph Neural Networks
• Poisoned classifiers are not only backdoored, they are fundamentally broken
• HaS-Nets: A Heal and Select Mechanism to Defend DNNs Against Backdoor Attacks for Data Collection Scenarios
• TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation
• Poisoning MorphNet for Clean-Label Backdoor Attack to Point Clouds
• Manipulating SGD with Data Ordering Attacks
• A Backdoor Attack against 3D Point Cloud Classifiers
• Hidden Backdoors in Human-Centric Language Models
• Black-box Detection of Backdoor Attacks with Limited Information and Data
• Regularization Can Help Mitigate Poisoning Attacks... with the Right Hyperparameters
• Incompatibility Clustering as a Defense Against Backdoor Poisoning Attacks