Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX

Abstract

With the advancement of machine learning (ML) and its growing awareness, many organizations who own data but not ML expertise (data owner) would like to pool their data and collaborate with those who have expertise but need data from diverse sources to train truly generalizable models (model owner). In such collaborative ML, the data owner wants to protect the privacy of its training data, while the model owner desires the confidentiality of the model and the training method which may contain intellectual properties. However, existing private ML solutions, such as federated learning and split learning, cannot meet the privacy requirements of both data and model owners at the same time. This paper presents Citadel, a scalable collaborative ML system that protects the privacy of both data owner and model owner in untrusted infrastructures with the help of Intel SGX. Citadel performs distributed training across multiple training enclaves running on behalf of data owners and an aggregator enclave on behalf of the model owner. Citadel further establishes a strong information barrier between these enclaves by means of zero-sum masking and hierarchical aggregation to prevent data/model leakage during collaborative training. Compared with the existing SGX-protected training systems, Citadel enables better scalability and stronger privacy guarantees for collaborative ML. Cloud deployment with various ML models shows that Citadel scales to a large number of enclaves with less than 1.73X slowdown caused by SGX.

References in corpus (22)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• ADADELTA: An Adaptive Learning Rate Method
• An overview of gradient descent optimization algorithms
• How To Backdoor Federated Learning
• Learning Differentially Private Recurrent Language Models
• Secure Federated Transfer Learning
• Differentially Private Federated Learning: A Client Level Perspective
• Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization
• Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
• Don't Use Large Mini-Batches, Use Local SGD
• SecureBoost: A Lossless Federated Learning Framework
• Split learning for health: Distributed deep learning without sharing raw patient data
• Chiron: Privacy-preserving Machine Learning as a Service
• Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization
• PipeDream: Fast and Efficient Pipeline Parallel DNN Training
• PrivFL: Practical Privacy-preserving Federated Regressions on High-dimensional Data over Mobile Networks
• Efficient Deep Learning on Multi-Source Private Data
• secureTF: A Secure TensorFlow Framework
• TensorSCONE: A Secure TensorFlow Framework using Intel SGX
• Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• On the Generalization Benefit of Noise in Stochastic Gradient Descent

Cited by in corpus (2)

• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective