198 citations · 764 across the 28 of their papers we have counts for
3 papers · 2 filters
ATOMO: Communication-efficient Learning via Atomic Sparsification
Hongyi Wang, Scott Sievert, Zachary Charles +3
Distributed model training suffers from communication overheads due to frequent gradient updates transmitted between compute nodes. To mitigate these overheads, several studies pro…
Gradient Coding via the Stochastic Block Model
Zachary Charles, Dimitris Papailiopoulos
Gradient descent and its many variants, including mini-batch stochastic gradient descent, form the algorithmic foundation of modern large-scale machine learning. Due to the size an…
DRACO: Byzantine-resilient Distributed Training via Redundant Gradients
Lingjiao Chen, Hongyi Wang, Zachary Charles +1
Distributed model training is vulnerable to byzantine system failures and adversarial compute nodes, i.e., nodes that use malicious updates to corrupt the global model stored at a…