Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation
arXiv:1903.03936
Abstract
Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning. The Byzantine model captures workers that behave arbitrarily, including malicious and compromised workers. In this paper, we break two prevailing Byzantine-tolerant techniques. Specifically we show robust aggregation methods for synchronous SGD -- coordinate-wise median and Krum -- can be broken using new attack strategies based on inner product manipulation. We prove our results theoretically, as well as show empirical validation.
References in corpus (5)
Cited by in corpus (17)
- Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks
- A Field Guide to Federated Optimization
- Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging
- Mitigating Backdoor Attacks in Federated Learning
- Secure Byzantine-Robust Machine Learning
- Learning from History for Byzantine Robust Optimization
- Byzantine-Resilient Non-Convex Stochastic Gradient Descent
- Robust Federated Recommendation System
- Federated Over-Air Subspace Tracking from Incomplete and Corrupted Data
- DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation
- ByGARS: Byzantine SGD with Arbitrary Number of Attackers
- Byzantine-Robust Variance-Reduced Federated Learning over Distributed Non-i.i.d. Data
- Secure Byzantine-Robust Distributed Learning via Clustering
- Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outliers
- Byzantine Resilient Distributed Multi-Task Learning
- Fast Machine Learning with Byzantine Workers and Servers
- Data Poisoning Attacks and Defenses to Crowdsourcing Systems