Federated Learning in Adversarial Settings
arXiv:2010.07808
Abstract
Federated Learning enables entities to collaboratively learn a shared prediction model while keeping their training data locally. It prevents data collection and aggregation and, therefore, mitigates the associated privacy risks. However, it still remains vulnerable to various security attacks where malicious participants aim at degrading the generated model, inserting backdoors, or inferring other participants' training data. This paper presents a new federated learning scheme that provides different trade-offs between robustness, privacy, bandwidth efficiency, and model accuracy. Our scheme uses biased quantization of model updates and hence is bandwidth efficient. It is also robust against state-of-the-art backdoor as well as model degradation attacks even when a large proportion of the participant nodes are malicious. We propose a practical differentially private extension of this scheme which protects the whole dataset of participating entities. We show that this extension performs as efficiently as the non-private but robust scheme, even with stringent privacy requirements but are less robust against model degradation and backdoor attacks. This suggests a possible fundamental trade-off between Differential Privacy and robustness.
References in corpus (19)
- Adam: A Method for Stochastic Optimization
- Deep Learning with Differential Privacy
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning: Strategies for Improving Communication Efficiency
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
- How To Backdoor Federated Learning
- Learning Differentially Private Recurrent Language Models
- TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
- Differentially Private Federated Learning: A Client Level Perspective
- Analyzing Federated Learning through an Adversarial Lens
- ATOMO: Communication-efficient Learning via Atomic Sparsification
- Practical Secure Aggregation for Federated Learning on User-Held Data
- Differential Privacy-enabled Federated Learning for Sensitive Health Data
- signSGD: Compressed Optimisation for Non-Convex Problems
- Rényi Differential Privacy of the Sampled Gaussian Mechanism
- A Hybrid Approach to Privacy-Preserving Federated Learning