Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning

Abstract

In this paper, we investigate over-the-air model aggregation in a federated edge learning (FEEL) system. We introduce a Markovian probability model to characterize the intrinsic temporal structure of the model aggregation series. With this temporal probability model, we formulate the model aggregation problem as to infer the desired aggregated update given all the past observations from a Bayesian perspective. We develop a message passing based algorithm, termed temporal-structure-assisted gradient aggregation (TSA-GA), to fulfil this estimation task with low complexity and near-optimal performance. We further establish the state evolution (SE) analysis to characterize the behaviour of the proposed TSA-GA algorithm, and derive an explicit bound of the expected loss reduction of the FEEL system under certain standard regularity conditions. In addition, we develop an expectation maximization (EM) strategy to learn the unknown parameters in the Markovian model. We show that the proposed TSAGA algorithm significantly outperforms the state-of-the-art, and is able to achieve comparable learning performance as the error-free benchmark in terms of both convergence rate and final test accuracy.

References in corpus (9)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Message Passing Algorithms for Compressed Sensing
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• On Analog Gradient Descent Learning over Multiple Access Fading Channels
• Turbo Compressed Sensing with Partial DFT Sensing Matrix
• Efficient High-Dimensional Inference in the Multiple Measurement Vector Problem
• On the Performance of Turbo Signal Recovery with Partial DFT Sensing Matrices
• Reconfigurable Intelligent Surface Enabled Federated Learning: A Unified Communication-Learning Design Approach

Cited by in corpus (1)

• Over-the-Air Federated Learning with Retransmissions (Extended Version)