A Survey on Fault-tolerance in Distributed Optimization and Machine Learning
arXiv:2106.08545
Abstract
The robustness of distributed optimization is an emerging field of study, motivated by various applications of distributed optimization including distributed machine learning, distributed sensing, and swarm robotics. With the rapid expansion of the scale of distributed systems, resilient distributed algorithms for optimization are needed, in order to mitigate system failures, communication issues, or even malicious attacks. This survey investigates the current state of fault-tolerance research in distributed optimization, and aims to provide an overview of the existing studies on both fault-tolerant distributed optimization theories and applicable algorithms.
35 pages, 1 figure, and 2 tables; Fixed a missing citation
References in corpus (26)
- Deep Learning with Differential Privacy
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Wide Residual Networks
- Robust Aggregation for Federated Learning
- The Hidden Vulnerability of Distributed Learning in Byzantium
- Generalized Byzantine-tolerant SGD
- Gradient Descent Converges to Minimizers
- Approximate Gradient Coding via Sparse Random Graphs
- Phocas: dimensional Byzantine-resilient stochastic gradient descent
- Secure Byzantine-Robust Machine Learning
- Byzantine Multi-Agent Optimization: Part II
- Learning from History for Byzantine Robust Optimization
- Byzantine-Tolerant Machine Learning
- Fault-Tolerant Distributed Optimization (Part IV): Constrained Optimization with Arbitrary Directed Networks
- Byzantine Fault Tolerant Distributed Linear Regression
- Distributed Momentum for Byzantine-resilient Learning
- Byzantine Fault-Tolerant Distributed Machine Learning Using Stochastic Gradient Descent (SGD) and Norm-Based Comparative Gradient Elimination (CGE)
- Fault-Tolerant Multi-Agent Optimization: Part III
- Approximate Byzantine Fault-Tolerance in Distributed Optimization
- Resilience in Collaborative Optimization: Redundant and Independent Cost Functions
- Fast Multidimensional Asymptotic and Approximate Consensus
- Byzantine Fault-Tolerance in Peer-to-Peer Distributed Gradient-Descent
- Randomized Reactive Redundancy for Byzantine Fault-Tolerance in Parallelized Learning
- Byzantine Fault-Tolerance in Decentralized Optimization under Minimal Redundancy
- Asynchronous Distributed Optimization with Redundancy in Cost Functions
- Detection of Insider Attacks in Distributed Projected Subgradient Algorithms