Asynchronous Federated Learning on Heterogeneous Devices: A Survey

Abstract

Federated learning (FL) is a kind of distributed machine learning framework, where the global model is generated on the centralized aggregation server based on the parameters of local models, addressing concerns about privacy leakage caused by the collection of local training data. With the growing computational and communication capacities of edge and IoT devices, applying FL on heterogeneous devices to train machine learning models is becoming a prevailing trend. Nonetheless, the synchronous aggregation strategy in the classic FL paradigm, particularly on heterogeneous devices, encounters limitations in resource utilization due to the need to wait for slow devices before aggregation in each training round. Furthermore, the uneven distribution of data across devices (i.e. data heterogeneity) in real-world scenarios adversely impacts the accuracy of the global model. Consequently, many asynchronous FL (AFL) approaches have been introduced across various application contexts to enhance efficiency, performance, privacy, and security. This survey comprehensively analyzes and summarizes existing AFL variations using a novel classification scheme, including device heterogeneity, data heterogeneity, privacy, and security on heterogeneous devices, as well as applications on heterogeneous devices. Finally, this survey reveals rising challenges and presents potentially promising research directions in this under-investigated domain.

References in corpus (17)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Pointer Sentinel Mixture Models
• A Secure Federated Learning Framework for 5G Networks
• Threats to Federated Learning: A Survey
• Individual Differential Privacy: A Utility-Preserving Formulation of Differential Privacy Guarantees
• VAFL: a Method of Vertical Asynchronous Federated Learning
• CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework
• Federated Learning with Buffered Asynchronous Aggregation
• FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
• Asynchronous Federated Learning with Differential Privacy for Edge Intelligence
• Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise
• Accurate and Fast Federated Learning via IID and Communication-Aware Grouping
• A Vertical Federated Learning Framework for Horizontally Partitioned Labels
• Real-time End-to-End Federated Learning: An Automotive Case Study
• Device Scheduling and Update Aggregation Policies for Asynchronous Federated Learning
• Accelerating Federated Learning in Heterogeneous Data and Computational Environments

Cited by in corpus (4)

• Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey
• An Efficient and Reliable Asynchronous Federated Learning Scheme for Smart Public Transportation
• Papaya: Practical, Private, and Scalable Federated Learning
• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques