A Little Is Enough: Circumventing Defenses For Distributed Learning

Abstract

Distributed learning is central for large-scale training of deep-learning models. However, they are exposed to a security threat in which Byzantine participants can interrupt or control the learning process. Previous attack models and their corresponding defenses assume that the rogue participants are (a) omniscient (know the data of all other participants), and (b) introduce large change to the parameters. We show that small but well-crafted changes are sufficient, leading to a novel non-omniscient attack on distributed learning that go undetected by all existing defenses. We demonstrate our attack method works not only for preventing convergence but also for repurposing of the model behavior (backdooring). We show that 20% of corrupt workers are sufficient to degrade a CIFAR10 model accuracy by 50%, as well as to introduce backdoors into MNIST and CIFAR10 models without hurting their accuracy

References in corpus (9)

• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• How To Backdoor Federated Learning
• Poisoning Attacks against Support Vector Machines
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• The Hidden Vulnerability of Distributed Learning in Byzantium
• Spectral Signatures in Backdoor Attacks
• Mitigating Sybils in Federated Learning Poisoning
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Generalized Byzantine-tolerant SGD

Cited by in corpus (21)

• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
• Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
• Learning from History for Byzantine Robust Optimization
• Backdoor Attacks on Federated Meta-Learning
• Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradients
• Robust Federated Recommendation System
• Byzantine-Resilient Non-Convex Stochastic Gradient Descent
• Distributed Momentum for Byzantine-resilient Learning
• ByGARS: Byzantine SGD with Arbitrary Number of Attackers
• Mitigating Sybil Attacks on Differential Privacy based Federated Learning
• Byzantine-resilient Decentralized Stochastic Gradient Descent
• Byzantine Resilient Distributed Multi-Task Learning
• Robust Federated Learning with Attack-Adaptive Aggregation
• Holdout SGD: Byzantine Tolerant Federated Learning
• Simeon -- Secure Federated Machine Learning Through Iterative Filtering
• ByzShield: An Efficient and Robust System for Distributed Training
• Data Poisoning Attacks and Defenses to Crowdsourcing Systems
• Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design
• On Provable Backdoor Defense in Collaborative Learning
• PipAttack: Poisoning Federated Recommender Systems forManipulating Item Promotion
• Probabilistic Inference for Learning from Untrusted Sources