Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation

Abstract

There is an increasing interest in a fast-growing machine learning technique called Federated Learning, in which the model training is distributed over mobile user equipments (UEs), exploiting UEs' local computation and training data. Despite its advantages in data privacy-preserving, Federated Learning (FL) still has challenges in heterogeneity across UEs' data and physical resources. We first propose a FL algorithm which can handle the heterogeneous UEs' data challenge without further assumptions except strongly convex and smooth loss functions. We provide the convergence rate characterizing the trade-off between local computation rounds of UE to update its local model and global communication rounds to update the FL global model. We then employ the proposed FL algorithm in wireless networks as a resource allocation optimization problem that captures the trade-off between the FL convergence wall clock time and energy consumption of UEs with heterogeneous computing and power resources. Even though the wireless resource allocation problem of FL is non-convex, we exploit this problem's structure to decompose it into three sub-problems and analyze their closed-form solutions as well as insights to problem design. Finally, we illustrate the theoretical analysis for the new algorithm with Tensorflow experiments and extensive numerical results for the wireless resource allocation sub-problems. The experiment results not only verify the theoretical convergence but also show that our proposed algorithm outperforms the vanilla FedAvg algorithm in terms of convergence rate and testing accuracy.

References in corpus (5)

• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Parallel and Distributed Methods for Nonconvex Optimization--Part II: Applications
• A Crowdsourcing Framework for On-Device Federated Learning
• On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization

Cited by in corpus (20)

• Edge Learning for B5G Networks with Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing
• Federated Learning: A Signal Processing Perspective
• A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guarantee
• Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks
• A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services
• Interference Management for Over-the-Air Federated Learning in Multi-Cell Wireless Networks
• STAR-RIS Integrated Non-Orthogonal Multiple Access and Over-the-Air Federated Learning: Framework, Analysis, and Optimization
• Federated Learning with Nesterov Accelerated Gradient
• Edge-assisted Democratized Learning Towards Federated Analytics
• AnycostFL: Efficient On-Demand Federated Learning over Heterogeneous Edge Devices
• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques
• Social-Aware Clustered Federated Learning with Customized Privacy Preservation
• Multi-Edge Server-Assisted Dynamic Federated Learning with an Optimized Floating Aggregation Point
• Towards Communication-Learning Trade-off for Federated Learning at the Network Edge
• Contrastive encoder pre-training-based clustered federated learning for heterogeneous data
• Gradient Sparsification for Efficient Wireless Federated Learning with Differential Privacy
• Federated Split Learning with Model Pruning and Gradient Quantization in Wireless Networks
• Federated Learning on Non-iid Data via Local and Global Distillation
• Egret: Reinforcement Mechanism for Sequential Computation Offloading in Edge Computing
• Communication Efficiency Optimization of Federated Learning for Computing and Network Convergence of 6G Networks