Confederated Learning: Federated Learning with Decentralized Edge Servers

Abstract

Federated learning (FL) is an emerging machine learning paradigm that allows to accomplish model training without aggregating data at a central server. Most studies on FL consider a centralized framework, in which a single server is endowed with a central authority to coordinate a number of devices to perform model training in an iterative manner. Due to stringent communication and bandwidth constraints, such a centralized framework has limited scalability as the number of devices grows. To address this issue, in this paper, we propose a ConFederated Learning (CFL) framework. The proposed CFL consists of multiple servers, in which each server is connected with an individual set of devices as in the conventional FL framework, and decentralized collaboration is leveraged among servers to make full use of the data dispersed throughout the network. We develop an alternating direction method of multipliers (ADMM) algorithm for CFL. The proposed algorithm employs a random scheduling policy which randomly selects a subset of devices to access their respective servers at each iteration, thus alleviating the need of uploading a huge amount of information from devices to servers. Theoretical analysis is presented to justify the proposed method. Numerical results show that the proposed method can converge to a decent solution significantly faster than gradient-based FL algorithms, thus boasting a substantial advantage in terms of communication efficiency.

arXiv comment13 pages, 5 figures

References in corpus (11)

• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Learning for Internet of Things: A Comprehensive Survey
• On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
• Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data
• On the Convergence of Local Descent Methods in Federated Learning
• Peer-to-peer Federated Learning on Graphs
• Variance-Reduced Decentralized Stochastic Optimization with Accelerated Convergence
• Variance Reduced Local SGD with Lower Communication Complexity
• FedHybrid: A Hybrid Primal-Dual Algorithm Framework for Federated Optimization
• Communication-Efficient ADMM-based Federated Learning

Cited by in corpus (2)

• Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges
• Collaborative Distributed Machine Learning