Widen The Backdoor To Let More Attackers In
arXiv:2110.04571
Abstract
As collaborative learning and the outsourcing of data collection become more common, malicious actors (or agents) which attempt to manipulate the learning process face an additional obstacle as they compete with each other. In backdoor attacks, where an adversary attempts to poison a model by introducing malicious samples into the training data, adversaries have to consider that the presence of additional backdoor attackers may hamper the success of their own backdoor. In this paper, we investigate the scenario of a multi-agent backdoor attack, where multiple non-colluding attackers craft and insert triggered samples in a shared dataset which is used by a model (a defender) to learn a task. We discover a clear backfiring phenomenon: increasing the number of attackers shrinks each attacker's attack success rate (ASR). We then exploit this phenomenon to minimize the collective ASR of attackers and maximize defender's robustness accuracy by (i) artificially augmenting the number of attackers, and (ii) indexing to remove the attacker's sub-dataset from the model for inference, hence proposing 2 defenses.
References in corpus (17)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Understanding Black-box Predictions via Influence Functions
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
- Poisoning Attacks against Support Vector Machines
- Can You Really Backdoor Federated Learning?
- Is feature selection secure against training data poisoning?
- Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
- Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
- Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
- Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
- Biometric Backdoors: A Poisoning Attack Against Unsupervised Template Updating
- Contamination Attacks and Mitigation in Multi-Party Machine Learning
- Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
- Dynamic backdoor attacks against federated learning
- Universal Multi-Party Poisoning Attacks
- Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
- Learning to Match Distributions for Domain Adaptation