Learning Rate Optimization for Federated Learning Exploiting Over-the-air Computation

Abstract

Federated learning (FL) as a promising edge-learning framework can effectively address the latency and privacy issues by featuring distributed learning at the devices and model aggregation in the central server. In order to enable efficient wireless data aggregation, over-the-air computation (AirComp) has recently been proposed and attracted immediate attention. However, fading of wireless channels can produce aggregate distortions in an AirComp-based FL scheme. To combat this effect, the concept of dynamic learning rate (DLR) is proposed in this work. We begin our discussion by considering multiple-input-single-output (MISO) scenario, since the underlying optimization problem is convex and has closed-form solution. We then extend our studies to more general multiple-input-multiple-output (MIMO) case and an iterative method is derived. Extensive simulation results demonstrate the effectiveness of the proposed scheme in reducing the aggregate distortion and guaranteeing the testing accuracy using the MNIST and CIFAR10 datasets. In addition, we present the asymptotic analysis and give a near-optimal receive beamforming design solution in closed form, which is verified by numerical simulations.

References in corpus (13)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• An overview of gradient descent optimization algorithms
• What should 6G be?
• Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Sparse Communication for Distributed Gradient Descent
• Federated Learning for 6G Communications: Challenges, Methods, and Future Directions
• Wireless Communications in the Era of Big Data
• True-data Testbed for 5G/B5G Intelligent Network
• Over-the-Air Computing for Wireless Data Aggregation in Massive IoT
• Federated Learning in Multi-RIS Aided Systems

Cited by in corpus (3)

• Federated Learning in Multi-RIS Aided Systems
• Optimized Power Control Design for Over-the-Air Federated Edge Learning
• Integrated user scheduling and beam steering in over-the-air federated learning for mobile IoT