On Analog Gradient Descent Learning over Multiple Access Fading Channels

Abstract

We consider a distributed learning problem over multiple access channel (MAC) using a large wireless network. The computation is made by the network edge and is based on received data from a large number of distributed nodes which transmit over a noisy fading MAC. The objective function is a sum of the nodes' local loss functions. This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning. We develop a novel Gradient-Based Multiple Access (GBMA) algorithm to solve the distributed learning problem over MAC. Specifically, the nodes transmit an analog function of the local gradient using common shaping waveforms and the network edge receives a superposition of the analog transmitted signals used for updating the estimate. GBMA does not require power control or beamforming to cancel the fading effect as in other algorithms, and operates directly with noisy distorted gradients. We analyze the performance of GBMA theoretically, and prove that it can approach the convergence rate of the centralized gradient descent (GD) algorithm in large networks. Specifically, we establish a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient. Furthermore, we provide energy scaling laws for approaching the centralized convergence rate as the number of nodes increases. Finally, experimental results support the theoretical findings, and demonstrate strong performance of GBMA using synthetic and real data.

arXiv comment32 pages, 6 figures

References in corpus (10)

• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Communication Efficient Federated Learning over Multiple Access Channels
• Federated Learning via Over-the-Air Computation
• Energy-Efficient Radio Resource Allocation for Federated Edge Learning
• Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)
• A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method
• Competing with the Empirical Risk Minimizer in a Single Pass
• Spectrum and Energy Efficient Multiple Access for Detection in Wireless Sensor Networks
• Exploiting Spatial Correlation in Energy Constrained Distributed Detection

Cited by in corpus (30)

• Over-the-Air Federated Learning from Heterogeneous Data
• Federated Learning via Intelligent Reflecting Surface
• Federated Learning: A Signal Processing Perspective
• Over-the-Air Computation for 6G: Foundations, Technologies, and Applications
• Communication Efficient Federated Learning over Multiple Access Channels
• Combating Interference for Over-the-Air Federated Learning: A Statistical Approach via RIS
• Over-the-Air Computing for Wireless Data Aggregation in Massive IoT
• Wireless Federated Learning with Local Differential Privacy
• Over-The-Air Computation in Correlated Channels
• Over-The-Air Computation in Correlated Channels
• Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs
• Anomaly Search with Multiple Plays under Delay and Switching Costs
• Riemannian Low-Rank Model Compression for Federated Learning with Over-the-Air Aggregation
• Optimized Power Control Design for Over-the-Air Federated Edge Learning
• Beamforming and Device Selection Design in Federated Learning with Over-the-air Aggregation
• Wireless for Machine Learning
• Privacy For Free: Wireless Federated Learning Via Uncoded Transmission With Adaptive Power Control
• Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning
• Fast Convergence Algorithm for Analog Federated Learning
• Delay Analysis of Wireless Federated Learning Based on Saddle Point Approximation and Large Deviation Theory
• Sparse Training for Federated Learning with Regularized Error Correction
• Over-the-Air Federated Learning with Retransmissions (Extended Version)
• Multi-Slot Over-The-Air Computation in Fading Channels
• Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach
• Optimizing Over-the-Air Computation in IRS-Aided C-RAN Systems
• Distributed Sparse SGD with Majority Voting
• Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications
• Byzantine-Resilient Federated Machine Learning via Over-the-Air Computation
• Over-the-Air Decentralized Federated Learning
• Channel-Adaptive Robust Aggregation for Over-the-Air Federated Learning in Heterogeneous Networks