Communication-Efficient Distributed SGD using Preamble-based Random Access
arXiv:2105.09427
Abstract
In this paper, we study communication-efficient distributed stochastic gradient descent (SGD) with data sets of users distributed over a certain area and communicating through wireless channels. Since the time for one iteration in the proposed approach is independent of the number of users, it is well-suited to scalable distributed SGD. Furthermore, since the proposed approach is based on preamble-based random access, which is widely adopted for machine-type communication (MTC), it can be easily employed for training models with a large number of devices in various Internet-of-Things (IoT) applications where MTC is used for their connectivity. For fading channel, we show that noncoherent combining can be used. As a result, no channel state information (CSI) estimation is required. From analysis and simulation results, we can confirm that the proposed approach is not only scalable, but also provides improved performance as the number of devices increases.
arXiv comment11 pages, 5 figures
References in corpus (6)
- Federated Optimization: Distributed Machine Learning for On-Device Intelligence
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
- Massive Machine-type Communications in 5G: Physical and MAC-layer solutions
- Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
- Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks
- IoT Connectivity Technologies and Applications: A Survey