A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guarantee

Abstract

Federated learning allows multiple participants to collaboratively train an efficient model without exposing data privacy. However, this distributed machine learning training method is prone to attacks from Byzantine clients, which interfere with the training of the global model by modifying the model or uploading the false gradient. In this paper, we propose a novel serverless federated learning framework Committee Mechanism based Federated Learning (CMFL), which can ensure the robustness of the algorithm with convergence guarantee. In CMFL, a committee system is set up to screen the uploaded local gradients. The committee system selects the local gradients rated by the elected members for the aggregation procedure through the selection strategy, and replaces the committee member through the election strategy. Based on the different considerations of model performance and defense, two opposite selection strategies are designed for the sake of both accuracy and robustness. Extensive experiments illustrate that CMFL achieves faster convergence and better accuracy than the typical Federated Learning, in the meanwhile obtaining better robustness than the traditional Byzantine-tolerant algorithms, in the manner of a decentralized approach. In addition, we theoretically analyze and prove the convergence of CMFL under different election and selection strategies, which coincides with the experimental results.

References in corpus (24)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Learning with Non-IID Data
• On the Convergence of FedAvg on Non-IID Data
• A Blockchain-based Decentralized Federated Learning Framework with Committee Consensus
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation
• LEAF: A Benchmark for Federated Settings
• Threats to Federated Learning: A Survey
• Stochastic Variance Reduction for Nonconvex Optimization
• Local SGD Converges Fast and Communicates Little
• BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning
• Learning to Detect Malicious Clients for Robust Federated Learning
• On the Convergence of Local Descent Methods in Federated Learning
• AdaGrad stepsizes: Sharp convergence over nonconvex landscapes
• Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging
• Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
• Peer-to-peer Federated Learning on Graphs
• Decentralized Federated Learning: A Segmented Gossip Approach
• Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies
• GFL: A Decentralized Federated Learning Framework Based On Blockchain
• Clustering Algorithm to Detect Adversaries in Federated Learning
• PPT: A Privacy-Preserving Global Model Training Protocol for Federated Learning in P2P Networks

Cited by in corpus (3)

• Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges
• Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations
• Decentralized Personalization for Federated Medical Image Segmentation via Gossip Contrastive Mutual Learning