PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments

Abstract

We propose and implement a Privacy-preserving Federated Learning ($PPFL$) framework for mobile systems to limit privacy leakages in federated learning. Leveraging the widespread presence of Trusted Execution Environments (TEEs) in high-end and mobile devices, we utilize TEEs on clients for local training, and on servers for secure aggregation, so that model/gradient updates are hidden from adversaries. Challenged by the limited memory size of current TEEs, we leverage greedy layer-wise training to train each model's layer inside the trusted area until its convergence. The performance evaluation of our implementation shows that $PPFL$ can significantly improve privacy while incurring small system overheads at the client-side. In particular, $PPFL$ can successfully defend the trained model against data reconstruction, property inference, and membership inference attacks. Furthermore, it can achieve comparable model utility with fewer communication rounds (0.54$\times$) and a similar amount of network traffic (1.002$\times$) compared to the standard federated learning of a complete model. This is achieved while only introducing up to ~15% CPU time, ~18% memory usage, and ~21% energy consumption overhead in $PPFL$'s client-side.

arXiv comment15 pages, 8 figures, accepted to MobiSys 2021

References in corpus (22)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• How To Backdoor Federated Learning
• Learning Differentially Private Recurrent Language Models
• Secure Federated Transfer Learning
• Revisiting Distributed Synchronous SGD
• Differentially Private Federated Learning: A Client Level Perspective
• Federated Optimization in Heterogeneous Networks
• Can You Really Backdoor Federated Learning?
• Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• DarkneTZ: Towards Model Privacy at the Edge using Trusted Execution Environments
• Chiron: Privacy-preserving Machine Learning as a Service
• White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
• Integrating Remote Attestation with Transport Layer Security
• Not one but many Tradeoffs: Privacy Vs. Utility in Differentially Private Machine Learning
• Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorization
• Layer-wise Characterization of Latent Information Leakage in Federated Learning
• Trust Anchors in Software Defined Networks
• L${}^2$-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks
• Policy-Based Federated Learning

Cited by in corpus (9)

• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Machine Learning with Confidential Computing: A Systematization of Knowledge
• Towards Battery-Free Machine Learning and Inference in Underwater Environments
• Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective
• PriMask: Cascadable and Collusion-Resilient Data Masking for Mobile Cloud Inference
• Personalised Federated Learning On Heterogeneous Feature Spaces
• Federated learning with differential privacy and an untrusted aggregator
• Separation of Powers in Federated Learning
• Security and Privacy Issues and Solutions in Federated Learning for Digital Healthcare