D2P-Fed: Differentially Private Federated Learning With Efficient Communication
arXiv:2006.13039
Abstract
In this paper, we propose the discrete Gaussian based differentially private federated learning (D2P-Fed), a unified scheme to achieve both differential privacy (DP) and communication efficiency in federated learning (FL). In particular, compared with the only prior work taking care of both aspects, D2P-Fed provides stronger privacy guarantee, better composability and smaller communication cost. The key idea is to apply the discrete Gaussian noise to the private data transmission. We provide complete analysis of the privacy guarantee, communication cost and convergence rate of D2P-Fed. We evaluated D2P-Fed on INFIMNIST and CIFAR10. The results show that D2P-Fed outperforms the-state-of-the-art by 4.7% to 13.0% in terms of model accuracy while saving one third of the communication cost.
References in corpus (10)
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- Renyi Differential Privacy
- Learning Differentially Private Recurrent Language Models
- Differentially Private Federated Learning: A Client Level Perspective
- Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences
- Distributed Mean Estimation with Limited Communication
- Communication Complexity of Distributed Convex Learning and Optimization
- The Discrete Gaussian for Differential Privacy
- Communication-Optimal Distributed Clustering
Cited by in corpus (7)
- DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation
- The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation
- Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy
- The Effect of Quantization in Federated Learning: A Rényi Differential Privacy Perspective
- Federated Learning with Local Differential Privacy: Trade-offs between Privacy, Utility, and Communication
- Differential Privacy Meets Federated Learning under Communication Constraints
- Prior-Independent Auctions for the Demand Side of Federated Learning