Analyzing Federated Learning through an Adversarial Lens

Abstract

Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server. In this work, we explore the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence. We explore a number of strategies to carry out this attack, starting with simple boosting of the malicious agent's update to overcome the effects of other agents' updates. To increase attack stealth, we propose an alternating minimization strategy, which alternately optimizes for the training loss and the adversarial objective. We follow up by using parameter estimation for the benign agents' updates to improve on attack success. Finally, we use a suite of interpretability techniques to generate visual explanations of model decisions for both benign and malicious models and show that the explanations are nearly visually indistinguishable. Our results indicate that even a highly constrained adversary can carry out model poisoning attacks while simultaneously maintaining stealth, thus highlighting the vulnerability of the federated learning setting and the need to develop effective defense strategies.

Extended version of paper accepted to ICML 2019, code available at https://github.com/inspire-group/ModelPoisoning; 19 pages, 14 figures

References in corpus (14)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Axiomatic Attribution for Deep Networks
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Understanding Black-box Predictions via Influence Functions
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• How To Backdoor Federated Learning
• SmoothGrad: removing noise by adding noise
• Poisoning Attacks against Support Vector Machines
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Sanity Checks for Saliency Maps
• The Hidden Vulnerability of Distributed Learning in Byzantium
• Is feature selection secure against training data poisoning?

Cited by in corpus (22)

• How To Backdoor Federated Learning
• Mitigating Sybils in Federated Learning Poisoning
• Threats to Federated Learning: A Survey
• Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
• Privacy Preserving Vertical Federated Learning for Tree-based Models
• Flower: A Friendly Federated Learning Research Framework
• FedRecAttack: Model Poisoning Attack to Federated Recommendation
• Free-riders in Federated Learning: Attacks and Defenses
• Can Adversarial Weight Perturbations Inject Neural Backdoors?
• FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling
• Towards Communication-efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things
• SAFELearning: Enable Backdoor Detectability In Federated Learning With Secure Aggregation
• Challenges of Privacy-Preserving Machine Learning in IoT
• Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learning
• Securing Federated Sensitive Topic Classification against Poisoning Attacks
• FedDebug: Systematic Debugging for Federated Learning Applications
• Dynamic backdoor attacks against federated learning
• Free-Rider Games for Federated Learning with Selfish Clients in NextG Wireless Networks
• Certifiably-Robust Federated Adversarial Learning via Randomized Smoothing
• Federated Learning in Adversarial Settings
• Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization
• Security and Privacy Issues and Solutions in Federated Learning for Digital Healthcare