Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation

Abstract

Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning. The Byzantine model captures workers that behave arbitrarily, including malicious and compromised workers. In this paper, we break two prevailing Byzantine-tolerant techniques. Specifically we show robust aggregation methods for synchronous SGD -- coordinate-wise median and Krum -- can be broken using new attack strategies based on inner product manipulation. We prove our results theoretically, as well as show empirical validation.

References in corpus (1)

• Distributed Robust Learning

Cited by in corpus (10)

• A Field Guide to Federated Optimization
• Mitigating Backdoor Attacks in Federated Learning
• Learning from History for Byzantine Robust Optimization
• Robust Federated Recommendation System
• Byzantine-Resilient Non-Convex Stochastic Gradient Descent
• Secure Byzantine-Robust Distributed Learning via Clustering
• Byzantine-Robust Variance-Reduced Federated Learning over Distributed Non-i.i.d. Data
• Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outliers
• Byzantine Resilient Distributed Multi-Task Learning
• Data Poisoning Attacks and Defenses to Crowdsourcing Systems