Distributed Additive Encryption and Quantization for Privacy Preserving Federated Deep Learning
arXiv:2011.12623
Abstract
Homomorphic encryption is a very useful gradient protection technique used in privacy preserving federated learning. However, existing encrypted federated learning systems need a trusted third party to generate and distribute key pairs to connected participants, making them unsuited for federated learning and vulnerable to security risks. Moreover, encrypting all model parameters is computationally intensive, especially for large machine learning models such as deep neural networks. In order to mitigate these issues, we develop a practical, computationally efficient encryption based protocol for federated deep learning, where the key pairs are collaboratively generated without the help of a third party. By quantization of the model parameters on the clients and an approximated aggregation on the server, the proposed method avoids encryption and decryption of the entire model. In addition, a threshold based secret sharing technique is designed so that no one can hold the global private key for decryption, while aggregated ciphertexts can be successfully decrypted by a threshold number of clients even if some clients are offline. Our experimental results confirm that the proposed method significantly reduces the communication costs and computational complexity compared to existing encrypted federated learning without compromising the performance and security.
References in corpus (14)
- Deep Learning in Neural Networks: An Overview
- Deep Learning with Differential Privacy
- Trained Ternary Quantization
- TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
- Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
- Differentially Private Federated Learning: A Client Level Perspective
- HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning
- LEAF: A Benchmark for Federated Settings
- Inverting Gradients -- How easy is it to break privacy in federated learning?
- High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning
- Federated Learning With Quantized Global Model Updates
- Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning
- Real-time Federated Evolutionary Neural Architecture Search
- From Federated Learning to Federated Neural Architecture Search: A Survey