No Peek: A Survey of private distributed deep learning

Abstract

We survey distributed deep learning models for training or inference without accessing raw data from clients. These methods aim to protect confidential patterns in data while still allowing servers to train models. The distributed deep learning methods of federated learning, split learning and large batch stochastic gradient descent are compared in addition to private and secure approaches of differential privacy, homomorphic encryption, oblivious transfer and garbled circuits in the context of neural networks. We study their benefits, limitations and trade-offs with regards to computational resources, data leakage and communication efficiency and also share our anticipated future trends.

arXiv comment21 pages

References in corpus (7)

• How To Backdoor Federated Learning
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Differentially Private Generative Adversarial Network
• Gazelle: A Low Latency Framework for Secure Neural Network Inference
• Split learning for health: Distributed deep learning without sharing raw patient data
• A Review of Homomorphic Encryption Libraries for Secure Computation
• Integral Privacy for Sampling

Cited by in corpus (20)

• A Survey on Distributed Machine Learning
• Precision Health Data: Requirements, Challenges and Existing Techniques for Data Security and Privacy
• Split Learning for collaborative deep learning in healthcare
• Artificial Intelligence in Glioma Imaging: Challenges and Advances
• End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data
• Distilling On-Device Intelligence at the Network Edge
• Practical Defences Against Model Inversion Attacks for Split Neural Networks
• FedCCEA : A Practical Approach of Client Contribution Evaluation for Federated Learning
• Defending against Reconstruction Attack in Vertical Federated Learning
• Federated Learning on Non-IID Data: A Survey
• ExpertMatcher: Automating ML Model Selection for Clients using Hidden Representations
• ExpertMatcher: Automating ML Model Selection for Users in Resource Constrained Countries
• Privacy Preserving for Medical Image Analysis via Non-Linear Deformation Proxy
• Digital Landscape of COVID-19 Testing: Challenges and Opportunities
• Privacy-Preserving Eye-tracking Using Deep Learning
• A Multisite, Report-Based, Centralized Infrastructure for Feedback and Monitoring of Radiology AI/ML Development and Clinical Deployment
• Gain without Pain: Offsetting DP-injected Nosies Stealthily in Cross-device Federated Learning
• Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design
• Data Markets to support AI for All: Pricing, Valuation and Governance