Joint Optimization of Communications and Federated Learning Over the Air

Abstract

Federated learning (FL) is an attractive paradigm for making use of rich distributed data while protecting data privacy. Nonetheless, nonideal communication links and limited transmission resources may hinder the implementation of fast and accurate FL. In this paper, we study joint optimization of communications and FL based on analog aggregation transmission in realistic wireless networks. We first derive closed-form expressions for the expected convergence rate of FL over the air, which theoretically quantify the impact of analog aggregation on FL. Based on the analytical results, we develop a joint optimization model for accurate FL implementation, which allows a parameter server to select a subset of workers and determine an appropriate power scaling factor. Since the practical setting of FL over the air encounters unobservable parameters, we reformulate the joint optimization of worker selection and power allocation using controlled approximation. Finally, we efficiently solve the resulting mixed-integer programming problem via a simple yet optimal finite-set search method by reducing the search space. Simulation results show that the proposed solutions developed for realistic wireless analog channels outperform a benchmark method, and achieve comparable performance of the ideal case where FL is implemented over noise-free wireless channels.

arXiv commentV2: Corrected typos and some technical content. Added some references. Added some theoretical proofs. Added some content to make our research applicable to more scenarios

References in corpus (14)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Challenges, Methods, and Future Directions
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Sparse Communication for Distributed Gradient Descent
• Communication Efficient Distributed Optimization using an Approximate Newton-type Method
• Hybrid Deterministic-Stochastic Methods for Data Fitting
• Sparsified SGD with Memory
• Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms
• Communication-Computation Efficient Gradient Coding
• DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
• On Maintaining Linear Convergence of Distributed Learning and Optimization under Limited Communication
• COKE: Communication-Censored Decentralized Kernel Learning

Cited by in corpus (5)

• Edge Learning for B5G Networks with Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing
• BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework
• Optimized Power Control Design for Over-the-Air Federated Edge Learning
• Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications