Local Model Poisoning Attacks to Byzantine-Robust Federated Learning

Abstract

In federated learning, multiple client devices jointly learn a machine learning model: each client device maintains a local model for its local training dataset, while a master device maintains a global model via aggregating the local models from the client devices. The machine learning community recently proposed several federated learning methods that were claimed to be robust against Byzantine failures (e.g., system failures, adversarial manipulations) of certain client devices. In this work, we perform the first systematic study on local model poisoning attacks to federated learning. We assume an attacker has compromised some client devices, and the attacker manipulates the local model parameters on the compromised client devices during the learning process such that the global model has a large testing error rate. We formulate our attacks as optimization problems and apply our attacks to four recent Byzantine-robust federated learning methods. Our empirical results on four real-world datasets show that our attacks can substantially increase the error rates of the models learnt by the federated learning methods that were claimed to be robust against Byzantine failures of some client devices. We generalize two defenses for data poisoning attacks to defend against our local model poisoning attacks. Our evaluation results show that one defense can effectively defend against our attacks in some cases, but the defenses are not effective enough in other cases, highlighting the need for new defenses against our local model poisoning attacks to federated learning.

arXiv commentAppeared in Usenix Security Symposium 2020. Fixed an error in Theorem 1. For demo code, see https://people.duke.edu/~zg70/code/fltrust.zip . For slides, see https://people.duke.edu/~zg70/code/Secure_Federated_Learning.pdf . For the talk, see https://www.youtube.com/watch?v=LP4uqW18yA0

References in corpus (20)

• Intriguing properties of neural networks
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Delving into Transferable Adversarial Examples and Black-box Attacks
• How To Backdoor Federated Learning
• Poisoning Attacks against Support Vector Machines
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• The Hidden Vulnerability of Distributed Learning in Byzantium
• Spectral Signatures in Backdoor Attacks
• On the (Statistical) Detection of Adversarial Examples
• Mitigating Sybils in Federated Learning Poisoning
• Is feature selection secure against training data poisoning?
• On Detecting Adversarial Perturbations
• Poisoning Attacks to Graph-Based Recommender Systems
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering

Cited by in corpus (13)

• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
• Learning to Detect Malicious Clients for Robust Federated Learning
• Poisoning Deep Learning Based Recommender Model in Federated Learning Scenarios
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradients
• Robust Federated Recommendation System
• Byzantine-Resilient Secure Federated Learning
• Mitigating Sybil Attacks on Differential Privacy based Federated Learning
• An Exploratory Analysis on Users' Contributions in Federated Learning
• Byzantine Resilient Distributed Multi-Task Learning
• Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization
• Towards Bidirectional Protection in Federated Learning
• Toward Smart Security Enhancement of Federated Learning Networks