PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy

Abstract

Federated Learning (FL) allows multiple participating clients to train machine learning models collaboratively by keeping their datasets local and only exchanging model updates. Existing FL protocol designs have been shown to be vulnerable to attacks that aim to compromise data privacy and/or model robustness. Recently proposed defenses focused on ensuring either privacy or robustness, but not both. In this paper, we develop a framework called PRECAD, which simultaneously achieves differential privacy (DP) and enhances robustness against model poisoning attacks with the help of cryptography. Using secure multi-party computation (MPC) techniques (e.g., secret sharing), noise is added to the model updates by the honest-but-curious server(s) (instead of each client) without revealing clients' inputs, which achieves the benefit of centralized DP in terms of providing a better privacy-utility tradeoff than local DP based solutions. Meanwhile, a crypto-aided secure validation protocol is designed to verify that the contribution of model update from each client is bounded without leaking privacy. We show analytically that the noise added to ensure DP also provides enhanced robustness against malicious model submissions. We experimentally demonstrate that our PRECAD framework achieves higher privacy-utility tradeoff and enhances robustness for the trained models.

arXiv commentarXiv admin note: text overlap with arXiv:2012.06337 by other authors

References in corpus (14)

• Deep Learning with Differential Privacy
• Renyi Differential Privacy
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Learning Differentially Private Recurrent Language Models
• Certified Adversarial Robustness via Randomized Smoothing
• Differentially Private Federated Learning: A Client Level Perspective
• Can You Really Backdoor Federated Learning?
• HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Prio: Private, Robust, and Scalable Computation of Aggregate Statistics
• CRFL: Certifiably Robust Federated Learning against Backdoor Attacks
• Secure Byzantine-Robust Machine Learning
• Differentially-Private "Draw and Discard" Machine Learning
• Secure Byzantine-Robust Distributed Learning via Clustering