Privacy-Preserving Deep Learning via Weight Transmission

Abstract

This paper considers the scenario that multiple data owners wish to apply a machine learning method over the combined dataset of all owners to obtain the best possible learning output but do not want to share the local datasets owing to privacy concerns. We design systems for the scenario that the stochastic gradient descent (SGD) algorithm is used as the machine learning method because SGD (or its variants) is at the heart of recent deep learning techniques over neural networks. Our systems differ from existing systems in the following features: {\bf (1)} any activation function can be used, meaning that no privacy-preserving-friendly approximation is required; {\bf (2)} gradients computed by SGD are not shared but the weight parameters are shared instead; and {\bf (3)} robustness against colluding parties even in the extreme case that only one honest party exists. We prove that our systems, while privacy-preserving, achieve the same learning accuracy as SGD and hence retain the merit of deep learning with respect to accuracy. Finally, we conduct several experiments using benchmark datasets, and show that our systems outperform previous system in terms of learning accuracies.

arXiv commentFull version of a conference paper at NSS 2017

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Deep Learning with Differential Privacy
• mixup: Beyond Empirical Risk Minimization
• Stealing Machine Learning Models via Prediction APIs
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications
• To Drop or Not to Drop: Robustness, Consistency and Differential Privacy Properties of Dropout

Cited by in corpus (8)

• Abnormal Client Behavior Detection in Federated Learning
• Trustworthy Privacy-preserving Hierarchical Ensemble and Federated Learning in Healthcare 4.0 with Blockchain
• A Privacy-Preserving Model based on Differential Approach for Sensitive Data in Cloud Environment
• A Federated Deep Learning Framework for Privacy Preservation and Communication Efficiency
• Privacy-Preserving Ensemble Infused Enhanced Deep Neural Network Framework for Edge Cloud Convergence
• Privacy-Preserving Blockchain Based Federated Learning with Differential Data Sharing
• Data Security and Privacy in Cloud Computing: Concepts and Emerging Trends
• Asyn2F: An Asynchronous Federated Learning Framework with Bidirectional Model Aggregation