Wireless Distributed Edge Learning: How Many Edge Devices Do We Need?
arXiv:2011.10894
Abstract
We consider distributed machine learning at the wireless edge, where a parameter server builds a global model with the help of multiple wireless edge devices that perform computations on local dataset partitions. Edge devices transmit the result of their computations (updates of current global model) to the server using a fixed rate and orthogonal multiple access over an error prone wireless channel. In case of a transmission error, the undelivered packet is retransmitted until successfully decoded at the receiver. Leveraging on the fundamental tradeoff between computation and communication in distributed systems, our aim is to derive how many edge devices are needed to minimize the average completion time while guaranteeing convergence. We provide upper and lower bounds for the average completion and we find a necessary condition for adding edge devices in two asymptotic regimes, namely the large dataset and the high accuracy regime. Conducted experiments on real datasets and numerical results confirm our analysis and substantiate our claim that the number of edge devices should be carefully selected for timely distributed edge learning.
arXiv commentIEEE Journal on Selected Areas in Communications (JSAC), accepted
References in corpus (10)
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning: Strategies for Improving Communication Efficiency
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
- Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
- Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
- A Joint Learning and Communications Framework for Federated Learning over Wireless Networks
- Communication Efficient Federated Learning over Multiple Access Channels
- Gradient Descent with Compressed Iterates
- An Introduction to Communication Efficient Edge Machine Learning
- Adaptive Federated Learning With Gradient Compression in Uplink NOMA