Mitigating Sybils in Federated Learning Poisoning

Abstract

Machine learning (ML) over distributed multi-party data is required for a variety of domains. Existing approaches, such as federated learning, collect the outputs computed by a group of devices at a central aggregator and run iterative algorithms to train a globally shared model. Unfortunately, such approaches are susceptible to a variety of attacks, including model poisoning, which is made substantially worse in the presence of sybils. In this paper we first evaluate the vulnerability of federated learning to sybil-based poisoning attacks. We then describe \emph{FoolsGold}, a novel defense to this problem that identifies poisoning sybils based on the diversity of client updates in the distributed learning process. Unlike prior work, our system does not bound the expected number of attackers, requires no auxiliary information outside of the learning process, and makes fewer assumptions about clients and their data. In our evaluation we show that FoolsGold exceeds the capabilities of existing state of the art approaches to countering sybil-based label-flipping and backdoor poisoning attacks. Our results hold for different distributions of client data, varying poisoning targets, and various sybil strategies. Code can be found at: https://github.com/DistributedML/FoolsGold

arXiv comment16 pages, Extended technical version of conference paper "The Limitations of Federated Learning in Sybil Settings" accepted at RAID 2020

References in corpus (9)

• Scikit-learn: Machine Learning in Python
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• How To Backdoor Federated Learning
• Poisoning Attacks against Support Vector Machines
• Differentially Private Federated Learning: A Client Level Perspective
• Analyzing Federated Learning through an Adversarial Lens
• Defending against Sybil Devices in Crowdsourced Mapping Services

Cited by in corpus (56)

• How To Backdoor Federated Learning
• From Distributed Machine Learning to Federated Learning: A Survey
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Threats to Federated Learning: A Survey
• Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
• Federated Reinforcement Learning: Techniques, Applications, and Open Challenges
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Split learning for health: Distributed deep learning without sharing raw patient data
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Edge Intelligence: Architectures, Challenges, and Applications
• FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning
• Free-riders in Federated Learning: Attacks and Defenses
• Federated Learning for Healthcare Informatics
• Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness
• Mitigating Backdoor Attacks in Federated Learning
• Attack-Resistant Federated Learning with Residual-based Reweighting
• Dynamic Defense Against Byzantine Poisoning Attacks in Federated Learning
• Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices
• Data Poisoning Attacks Against Federated Learning Systems
• A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspective
• Enhanced Security and Privacy via Fragmented Federated Learning
• Privacy-preserving Intelligent Resource Allocation for Federated Edge Learning in Quantum Internet
• A Little Is Enough: Circumventing Defenses For Distributed Learning
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• SAFELearning: Enable Backdoor Detectability In Federated Learning With Secure Aggregation
• A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning
• Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus
• Eavesdrop the Composition Proportion of Training Labels in Federated Learning
• Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
• A Federated Learning Approach for Mobile Packet Classification
• Backdoor Attacks on Federated Meta-Learning
• FedEval: A Holistic Evaluation Framework for Federated Learning
• Turning Federated Learning Systems Into Covert Channels
• Resisting Backdoor Attacks in Federated Learning via Bidirectional Elections and Individual Perspective
• Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection
• ARFED: Attack-Resistant Federated averaging based on outlier elimination
• Securing Federated Sensitive Topic Classification against Poisoning Attacks
• BlockFLA: Accountable Federated Learning via Hybrid Blockchain Architecture
• Hijack Vertical Federated Learning Models As One Party
• DistFL: Distribution-aware Federated Learning for Mobile Scenarios
• Wireless for Machine Learning
• Towards Practical Watermark for Deep Neural Networks in Federated Learning
• Mitigating Sybil Attacks on Differential Privacy based Federated Learning
• Reliable Federated Learning for Mobile Networks
• Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks
• On the Sample Complexity of Adversarial Multi-Source PAC Learning
• Robust Federated Learning with Attack-Adaptive Aggregation
• SLSGD: Secure and Efficient Distributed On-device Machine Learning
• An Exploratory Analysis on Users' Contributions in Federated Learning
• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
• Towards Bidirectional Protection in Federated Learning
• Reaching Data Confidentiality and Model Accountability on the CalTrain
• Toward Smart Security Enhancement of Federated Learning Networks
• A Separation Result Between Data-oblivious and Data-aware Poisoning Attacks
• BadVFL: Backdoor Attacks in Vertical Federated Learning
• Bristle: Decentralized Federated Learning in Byzantine, Non-i.i.d. Environments