SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead

Abstract

Federated learning (FL) has attracted increasing attention as a promising approach to driving a vast number of end devices with artificial intelligence. However, it is very challenging to guarantee the efficiency of FL considering the unreliable nature of end devices while the cost of device-server communication cannot be neglected. In this paper, we propose SAFA, a semi-asynchronous FL protocol, to address the problems in federated learning such as low round efficiency and poor convergence rate in extreme conditions (e.g., clients dropping offline frequently). We introduce novel designs in the steps of model distribution, client selection and global aggregation to mitigate the impacts of stragglers, crashes and model staleness in order to boost efficiency and improve the quality of the global model. We have conducted extensive experiments with typical machine learning tasks. The results demonstrate that the proposed protocol is effective in terms of shortening federated round duration, reducing local resource wastage, and improving the accuracy of the global model at an acceptable communication cost.

arXiv comment16 pages, 8 figures

References in corpus (4)

• Federated Learning: Strategies for Improving Communication Efficiency
• Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
• Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
• Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing

Cited by in corpus (21)

• Accelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systems
• Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data
• Resource-Efficient Federated Learning
• Artificial Intelligence of Things: A Survey
• An Efficient and Reliable Asynchronous Federated Learning Scheme for Smart Public Transportation
• Asynchronous Decentralized Federated Learning for Collaborative Fault Diagnosis of PV Stations
• Secure and Efficient Federated Learning Through Layering and Sharding Blockchain
• Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks
• GitFL: Adaptive Asynchronous Federated Learning using Version Control
• Semi-Synchronous Personalized Federated Learning over Mobile Edge Networks
• Towards Reliable Participation in UAV-Enabled Federated Edge Learning on Non-IID Data
• Federated Learning for Energy Constrained IoT devices: A systematic mapping study
• FedLesScan: Mitigating Stragglers in Serverless Federated Learning
• Asynchronous Online Federated Learning with Reduced Communication Requirements
• EchoPFL: Asynchronous Personalized Federated Learning on Mobile Devices with On-Demand Staleness Control
• FedSAUC: A Similarity-Aware Update Control for Communication-Efficient Federated Learning in Edge Computing
• Efficient, Portable, Census-Polymorphic Choreographic Programming
• Communication Efficiency Optimization of Federated Learning for Computing and Network Convergence of 6G Networks
• Accelerating MoE Model Inference with Expert Sharding
• Practical Federated Learning without a Server
• Efficiently Distributed Federated Learning