Communication Efficient Federated Learning with Energy Awareness over Wireless Networks

Abstract

In federated learning (FL), reducing the communication overhead is one of the most critical challenges since the parameter server and the mobile devices share the training parameters over wireless links. With such consideration, we adopt the idea of SignSGD in which only the signs of the gradients are exchanged. Moreover, most of the existing works assume Channel State Information (CSI) available at both the mobile devices and the parameter server, and thus the mobile devices can adopt fixed transmission rates dictated by the channel capacity. In this work, only the parameter server side CSI is assumed, and channel capacity with outage is considered. In this case, an essential problem for the mobile devices is to select appropriate local processing and communication parameters (including the transmission rates) to achieve a desired balance between the overall learning performance and their energy consumption. Two optimization problems are formulated and solved, which optimize the learning performance given the energy consumption requirement, and vice versa. Furthermore, considering that the data may be distributed across the mobile devices in a highly uneven fashion in FL, a stochastic sign-based algorithm is proposed. Extensive simulations are performed to demonstrate the effectiveness of the proposed methods.

References in corpus (12)

• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Towards Federated Learning at Scale: System Design
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• ATOMO: Communication-efficient Learning via Atomic Sparsification
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
• High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning
• Delay Minimization for Federated Learning Over Wireless Communication Networks
• Communication Efficient Federated Learning over Multiple Access Channels
• Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms