198 citations · 764 across the 28 of their papers we have counts for
6 papers · 1 filter
ATOMO: Communication-efficient Learning via Atomic Sparsification
Hongyi Wang, Scott Sievert, Zachary Charles +3
Distributed model training suffers from communication overheads due to frequent gradient updates transmitted between compute nodes. To mitigate these overheads, several studies pro…
Gradient Coding via the Stochastic Block Model
Zachary Charles, Dimitris Papailiopoulos
Gradient descent and its many variants, including mini-batch stochastic gradient descent, form the algorithmic foundation of modern large-scale machine learning. Due to the size an…
DRACO: Byzantine-resilient Distributed Training via Redundant Gradients
Lingjiao Chen, Hongyi Wang, Zachary Charles +1
Distributed model training is vulnerable to byzantine system failures and adversarial compute nodes, i.e., nodes that use malicious updates to corrupt the global model stored at a…
Approximate Gradient Coding via Sparse Random Graphs
Zachary Charles, Dimitris Papailiopoulos, Jordan Ellenberg
Distributed algorithms are often beset by the straggler effect, where the slowest compute nodes in the system dictate the overall running time. Coding-theoretic techniques have bee…
Stability and Generalization of Learning Algorithms that Converge to Global Optima
Zachary Charles, Dimitris Papailiopoulos
We establish novel generalization bounds for learning algorithms that converge to global minima. We do so by deriving black-box stability results that only depend on the convergenc…
Subspace Clustering with Missing and Corrupted Data
Zachary Charles, Amin Jalali, Rebecca Willett
Given full or partial information about a collection of points that lie close to a union of several subspaces, subspace clustering refers to the process of clustering the points ac…