Distributed Learning in Wireless Networks: Recent Progress and Future Challenges
arXiv:2104.02151
Abstract
The next-generation of wireless networks will enable many machine learning (ML) tools and applications to efficiently analyze various types of data collected by edge devices for inference, autonomy, and decision making purposes. However, due to resource constraints, delay limitations, and privacy challenges, edge devices cannot offload their entire collected datasets to a cloud server for centrally training their ML models or inference purposes. To overcome these challenges, distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges, thus reducing the communication overhead and latency as well as improving data privacy. However, deploying distributed learning over wireless networks faces several challenges including the uncertain wireless environment, limited wireless resources (e.g., transmit power and radio spectrum), and hardware resources. This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks. We present a detailed overview of several emerging distributed learning paradigms, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning. For each learning framework, we first introduce the motivation for deploying it over wireless networks. Then, we present a detailed literature review on the use of communication techniques for its efficient deployment. We then introduce an illustrative example to show how to optimize wireless networks to improve its performance. Finally, we introduce future research opportunities. In a nutshell, this paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks.
References in corpus (28)
- Deep Learning with Differential Privacy
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Language Models are Few-Shot Learners
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- BinaryConnect: Training Deep Neural Networks with binary weights during propagations
- Deep Learning with Limited Numerical Precision
- Compressing Deep Convolutional Networks using Vector Quantization
- Revisiting Distributed Synchronous SGD
- Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
- Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
- Training deep neural networks with low precision multiplications
- UVeQFed: Universal Vector Quantization for Federated Learning
- Sparsified SGD with Memory
- JALAD: Joint Accuracy- and Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution
- Error Feedback Fixes SignSGD and other Gradient Compression Schemes
- Fast Deep Learning for Automatic Modulation Classification
- Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks
- signSGD with Majority Vote is Communication Efficient And Fault Tolerant
- Joint Device-Edge Inference over Wireless Links with Pruning
- Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients
- Dopamine: Differentially Private Federated Learning on Medical Data
- Pruning Algorithms to Accelerate Convolutional Neural Networks for Edge Applications: A Survey
- SparseCast: Hybrid Digital-Analog Wireless Image Transmission Exploiting Frequency Domain Sparsity
- Over-the-Air Computing for Wireless Data Aggregation in Massive IoT
- Gradient Statistics Aware Power Control for Over-the-Air Federated Learning
- Sparse Communication for Training Deep Networks
- Improving Device-Edge Cooperative Inference of Deep Learning via 2-Step Pruning
- Distributed Multi-agent Meta Learning for Trajectory Design in Wireless Drone Networks