Federated Learning with Differential Privacy: Algorithms and Performance Analysis

Abstract

In this paper, to effectively prevent information leakage, we propose a novel framework based on the concept of differential privacy (DP), in which artificial noises are added to the parameters at the clients side before aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting different variances of artificial noises. Then we develop a theoretical convergence bound of the loss function of the trained FL model in the NbAFL. Specifically, the theoretical bound reveals the following three key properties: 1) There is a tradeoff between the convergence performance and privacy protection levels, i.e., a better convergence performance leads to a lower protection level; 2) Given a fixed privacy protection level, increasing the number $N$ of overall clients participating in FL can improve the convergence performance; 3) There is an optimal number of maximum aggregation times (communication rounds) in terms of convergence performance for a given protection level. Furthermore, we propose a $K$-random scheduling strategy, where $K$ ($1<K<N$) clients are randomly selected from the $N$ overall clients to participate in each aggregation. We also develop the corresponding convergence bound of the loss function in this case and the $K$-random scheduling strategy can also retain the above three properties. Moreover, we find that there is an optimal $K$ that achieves the best convergence performance at a fixed privacy level. Evaluations demonstrate that our theoretical results are consistent with simulations, thereby facilitating the designs on various privacy-preserving FL algorithms with different tradeoff requirements on convergence performance and privacy levels.

References in corpus (5)

• Deep Learning with Differential Privacy
• Federated Learning: Strategies for Improving Communication Efficiency
• In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning
• Differentially Private Federated Learning: A Client Level Perspective
• Differentially Private Meta-Learning

Cited by in corpus (11)

• Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey
• Privacy-preserving Artificial Intelligence Techniques in Biomedicine
• Local Differential Privacy and Its Applications: A Comprehensive Survey
• The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscape
• Wireless Federated Learning with Local Differential Privacy
• FLEdge: Benchmarking Federated Machine Learning Applications in Edge Computing Systems
• Federated Learning -- Methods, Applications and beyond
• Federated Edge Learning : Design Issues and Challenges
• Distributed Deep Learning in Open Collaborations
• A Differentially Private Probabilistic Framework for Modeling the Variability Across Federated Datasets of Heterogeneous Multi-View Observations
• FedOCR: Communication-Efficient Federated Learning for Scene Text Recognition