Voting-based Approaches For Differentially Private Federated Learning
arXiv:2010.04851
Abstract
Differentially Private Federated Learning (DPFL) is an emerging field with many applications. Gradient averaging based DPFL methods require costly communication rounds and hardly work with large-capacity models, due to the explicit dimension dependence in its added noise. In this work, inspired by knowledge transfer non-federated privacy learning from Papernot et al.(2017; 2018), we design two new DPFL schemes, by voting among the data labels returned from each local model, instead of averaging the gradients, which avoids the dimension dependence and significantly reduces the communication cost. Theoretically, by applying secure multi-party computation, we could exponentially amplify the (data-dependent) privacy guarantees when the margin of the voting scores are large. Extensive experiments show that our approaches significantly improve the privacy-utility trade-off over the state-of-the-arts in DPFL.
References in corpus (16)
- Deep Learning with Differential Privacy
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning: Strategies for Improving Communication Efficiency
- Renyi Differential Privacy
- Federated Learning for Mobile Keyboard Prediction
- On the Convergence of FedAvg on Non-IID Data
- Learning Differentially Private Recurrent Language Models
- The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
- Differentially Private Federated Learning: A Client Level Perspective
- Federated Optimization in Heterogeneous Networks
- Scalable Private Learning with PATE
- Agnostic Federated Learning
- Protection Against Reconstruction and Its Applications in Private Federated Learning
- Federated Adversarial Domain Adaptation
- Private Federated Learning with Domain Adaptation
- Fear Not, Vote Truthfully: Secure Multiparty Computation of Score Based Rules