Byzantine-Robust and Privacy-Preserving Framework for FedML
arXiv:2105.02295
Abstract
Federated learning has emerged as a popular paradigm for collaboratively training a model from data distributed among a set of clients. This learning setting presents, among others, two unique challenges: how to protect privacy of the clients' data during training, and how to ensure integrity of the trained model. We propose a two-pronged solution that aims to address both challenges under a single framework. First, we propose to create secure enclaves using a trusted execution environment (TEE) within the server. Each client can then encrypt their gradients and send them to verifiable enclaves. The gradients are decrypted within the enclave without the fear of privacy breaches. However, robustness check computations in a TEE are computationally prohibitive. Hence, in the second step, we perform a novel gradient encoding that enables TEEs to encode the gradients and then offloading Byzantine check computations to accelerators such as GPUs. Our proposed approach provides theoretical bounds on information leakage and offers a significant speed-up over the baseline in empirical evaluation.
References in corpus (10)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
- How To Backdoor Federated Learning
- Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
- The Hidden Vulnerability of Distributed Learning in Byzantium
- Inverting Gradients -- How easy is it to break privacy in federated learning?
- Dopamine: Differentially Private Federated Learning on Medical Data
- Secure multiparty computations in floating-point arithmetic