Privacy For Free: Wireless Federated Learning Via Uncoded Transmission With Adaptive Power Control

Abstract

Federated Learning (FL) refers to distributed protocols that avoid direct raw data exchange among the participating devices while training for a common learning task. This way, FL can potentially reduce the information on the local data sets that is leaked via communications. In order to provide formal privacy guarantees, however, it is generally necessary to put in place additional masking mechanisms. When FL is implemented in wireless systems via uncoded transmission, the channel noise can directly act as a privacy-inducing mechanism. This paper demonstrates that, as long as the privacy constraint level, measured via differential privacy (DP), is below a threshold that decreases with the signal-to-noise ratio (SNR), uncoded transmission achieves privacy "for free", i.e., without affecting the learning performance. More generally, this work studies adaptive power allocation (PA) for decentralized gradient descent in wireless FL with the aim of minimizing the learning optimality gap under privacy and power constraints. Both orthogonal multiple access (OMA) and non-orthogonal multiple access (NOMA) transmission with "over-the-air-computing" are studied, and solutions are obtained in closed form for an offline optimization setting. Furthermore, heuristic online methods are proposed that leverage iterative one-step-ahead optimization. The importance of dynamic PA and the potential benefits of NOMA versus OMA are demonstrated through extensive simulations.

Published in IEEE Journal on Selected Areas in Communications ( Volume: 39, Issue: 1, Jan. 2021) DOI: 10.1109/JSAC.2020.3036948

References in corpus (13)

• Federated Learning with Non-IID Data
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• On Analog Gradient Descent Learning over Multiple Access Fading Channels
• Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences
• High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning
• Understanding Gradient Clipping in Private SGD: A Geometric Perspective
• Communication Efficient Federated Learning over Multiple Access Channels
• Wireless Federated Learning with Local Differential Privacy
• Gradient Statistics Aware Power Control for Over-the-Air Federated Learning
• Efficient Federated Learning over Multiple Access Channel with Differential Privacy Constraints
• The Value of Collaboration in Convex Machine Learning with Differential Privacy

Cited by in corpus (2)

• Over-the-Air Federated Learning from Heterogeneous Data
• QoS-Constrained Federated Learning Empowered by Intelligent Reflecting Surface