To Talk or to Work: Flexible Communication Compression for Energy Efficient Federated Learning over Heterogeneous Mobile Edge Devices
arXiv:2012.11804
Abstract
Recent advances in machine learning, wireless communication, and mobile hardware technologies promisingly enable federated learning (FL) over massive mobile edge devices, which opens new horizons for numerous intelligent mobile applications. Despite the potential benefits, FL imposes huge communication and computation burdens on participating devices due to periodical global synchronization and continuous local training, raising great challenges to battery constrained mobile devices. In this work, we target at improving the energy efficiency of FL over mobile edge networks to accommodate heterogeneous participating devices without sacrificing the learning performance. To this end, we develop a convergence-guaranteed FL algorithm enabling flexible communication compression. Guided by the derived convergence bound, we design a compression control scheme to balance the energy consumption of local computing (i.e., "working") and wireless communication (i.e., "talking") from the long-term learning perspective. In particular, the compression parameters are elaborately chosen for FL participants adapting to their computing and communication environments. Extensive simulations are conducted using various datasets to validate our theoretical analysis, and the results also demonstrate the efficacy of the proposed scheme in energy saving.
Accepted for publication in INFOCOM'21
References in corpus (11)
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Federated Learning for Mobile Keyboard Prediction
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
- Parallel and Distributed Methods for Nonconvex Optimization--Part II: Applications
- Sparsified SGD with Memory
- Communication Compression for Decentralized Training
- On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization
- Convergence Time Optimization for Federated Learning over Wireless Networks
- On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization
- A flexible framework for communication-efficient machine learning: from HPC to IoT