Federated Learning via Over-the-Air Computation

Abstract

The stringent requirements for low-latency and privacy of the emerging high-stake applications with intelligent devices such as drones and smart vehicles make the cloud computing inapplicable in these scenarios. Instead, edge machine learning becomes increasingly attractive for performing training and inference directly at network edges without sending data to a centralized data center. This stimulates a nascent field termed as federated learning for training a machine learning model on computation, storage, energy and bandwidth limited mobile devices in a distributed manner. To preserve data privacy and address the issues of unbalanced and non-IID data points across different devices, the federated averaging algorithm has been proposed for global model aggregation by computing the weighted average of locally updated model at each selected device. However, the limited communication bandwidth becomes the main bottleneck for aggregating the locally computed updates. We thus propose a novel over-the-air computation based approach for fast global model aggregation via exploring the superposition property of a wireless multiple-access channel. This is achieved by joint device selection and beamforming design, which is modeled as a sparse and low-rank optimization problem to support efficient algorithms design. To achieve this goal, we provide a difference-of-convex-functions (DC) representation for the sparse and low-rank function to enhance sparsity and accurately detect the fixed-rank constraint in the procedure of device selection. A DC algorithm is further developed to solve the resulting DC program with global convergence guarantees. The algorithmic advantages and admirable performance of the proposed methodologies are demonstrated through extensive numerical results.

References in corpus (12)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning with Non-IID Data
• Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
• Towards Federated Learning at Scale: System Design
• An overview of low-rank matrix recovery from incomplete observations
• A Berkeley View of Systems Challenges for AI
• Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms
• An Efficient Global Algorithm for Single-Group Multicast Beamforming
• Towards an Intelligent Edge: Wireless Communication Meets Machine Learning
• Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)
• GIANT: Globally Improved Approximate Newton Method for Distributed Optimization
• Nonconvex and Nonsmooth Sparse Optimization via Adaptively Iterative Reweighted Methods

Cited by in corpus (18)

• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Multi-Armed Bandit Based Client Scheduling for Federated Learning
• On Analog Gradient Descent Learning over Multiple Access Fading Channels
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Federated Learning for Healthcare Informatics
• A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression
• Delay Minimization for Federated Learning Over Wireless Communication Networks
• Communication Efficient Federated Learning over Multiple Access Channels
• Wireless Federated Learning with Local Differential Privacy
• Intelligent Reflecting Surface for Downlink Non-Orthogonal Multiple Access Networks
• Wireless for Machine Learning
• Optimized Power Control for Over-the-Air Computation in Fading Channels
• Data-Importance Aware User Scheduling for Communication-Efficient Edge Machine Learning
• Byzantine Resilient Distributed Multi-Task Learning
• VoiceMoji: A Novel On-Device Pipeline for Seamless Emoji Insertion in Dictation
• Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach