Secure Byzantine-Robust Distributed Learning via Clustering
arXiv:2110.02940
Abstract
Federated learning systems that jointly preserve Byzantine robustness and privacy have remained an open problem. Robust aggregation, the standard defense for Byzantine attacks, generally requires server access to individual updates or nonlinear computation -- thus is incompatible with privacy-preserving methods such as secure aggregation via multiparty computation. To this end, we propose SHARE (Secure Hierarchical Robust Aggregation), a distributed learning framework designed to cryptographically preserve client update privacy and robustness to Byzantine adversaries simultaneously. The key idea is to incorporate secure averaging among randomly clustered clients before filtering malicious updates through robust aggregation. Experiments show that SHARE has similar robustness guarantees as existing techniques while enhancing privacy.
18 pages, 9 Figures
References in corpus (12)
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning: Strategies for Improving Communication Efficiency
- Federated Learning for Mobile Keyboard Prediction
- Towards Federated Learning at Scale: System Design
- Robust Aggregation for Federated Learning
- LEAF: A Benchmark for Federated Settings
- Robust Federated Learning in a Heterogeneous Environment
- Adaptive Federated Optimization
- Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance
- Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation
- Secure Byzantine-Robust Machine Learning
- Towards Bidirectional Protection in Federated Learning