A Federated Deep Learning Framework for Privacy Preservation and Communication Efficiency

Abstract

Deep learning has achieved great success in many applications. However, its deployment in practice has been hurdled by two issues: the privacy of data that has to be aggregated centrally for model training and high communication overhead due to transmission of a large amount of data usually geographically distributed. Addressing both issues is challenging and most existing works could not provide an efficient solution. In this paper, we develop FedPC, a Federated Deep Learning Framework for Privacy Preservation and Communication Efficiency. The framework allows a model to be learned on multiple private datasets while not revealing any information of training data, even with intermediate data. The framework also minimizes the amount of data exchanged to update the model. We formally prove the convergence of the learning model when training with FedPC and its privacy-preserving property. We perform extensive experiments to evaluate the performance of FedPC in terms of the approximation to the upper-bound performance (when training centrally) and communication overhead. The results show that FedPC maintains the performance approximation of the models within $8.5\%$ of the centrally-trained models when data is distributed to 10 computing nodes. FedPC also reduces the communication overhead by up to $42.20\%$ compared to existing works.

References in corpus (10)

• Deep Learning with Differential Privacy
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
• Federated Machine Learning: Concept and Applications
• Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• Scalable Private Learning with PATE
• Privacy-Preserving Deep Learning via Weight Transmission

Cited by in corpus (2)

• Asyn2F: An Asynchronous Federated Learning Framework with Bidirectional Model Aggregation
• SCoTTi: Save Computation at Training Time with an adaptive framework