Accelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systems

Abstract

Mobile Edge Computing (MEC), which incorporates the Cloud, edge nodes and end devices, has shown great potential in bringing data processing closer to the data sources. Meanwhile, Federated learning (FL) has emerged as a promising privacy-preserving approach to facilitating AI applications. However, it remains a big challenge to optimize the efficiency and effectiveness of FL when it is integrated with the MEC architecture. Moreover, the unreliable nature (e.g., stragglers and intermittent drop-out) of end devices significantly slows down the FL process and affects the global model's quality Xin such circumstances. In this paper, a multi-layer federated learning protocol called HybridFL is designed for the MEC architecture. HybridFL adopts two levels (the edge level and the cloud level) of model aggregation enacting different aggregation strategies. Moreover, in order to mitigate stragglers and end device drop-out, we introduce regional slack factors into the stage of client selection performed at the edge nodes using a probabilistic approach without identifying or probing the state of end devices (whose reliability is agnostic). We demonstrate the effectiveness of our method in modulating the proportion of clients selected and present the convergence analysis for our protocol. We have conducted extensive experiments with machine learning tasks in different scales of MEC system. The results show that HybridFL improves the FL training process significantly in terms of shortening the federated round length, speeding up the global model's convergence (by up to 12X) and reducing end device energy consumption (by up to 58%).

14 pages, 7 figures, with Appendix

References in corpus (10)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Mobile Edge Computing: A Survey on Architecture and Computation Offloading
• Federated Learning: Strategies for Improving Communication Efficiency
• HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
• Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
• Towards Federated Learning at Scale: System Design
• Asynchronous Federated Optimization
• SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead
• Asynchronous Stochastic Gradient Descent with Delay Compensation
• Client-Edge-Cloud Hierarchical Federated Learning

Cited by in corpus (8)

• Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence
• Analysis and Optimal Edge Assignment For Hierarchical Federated Learning on Non-IID Data
• Asynchronous Decentralized Federated Learning for Collaborative Fault Diagnosis of PV Stations
• Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies
• Towards Reliable Participation in UAV-Enabled Federated Edge Learning on Non-IID Data
• On the Impact of Device and Behavioral Heterogeneity in Federated Learning
• Towards Inference Delivery Networks: Distributing Machine Learning with Optimality Guarantees
• FedProf: Selective Federated Learning with Representation Profiling