885 citations · 2.1k across the 60 of their papers we have counts for
6 papers · 1 filter
DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation
Shashank Rajput, Hongyi Wang, Zachary Charles +1
To improve the resilience of distributed training to worst-case, or Byzantine node failures, several recent approaches have replaced gradient averaging with robust aggregation meth…
Bad Global Minima Exist and SGD Can Reach Them
Shengchao Liu, Dimitris Papailiopoulos, Dimitris Achlioptas
Several works have aimed to explain why overparameterized neural networks generalize well when trained by Stochastic Gradient Descent (SGD). The consensus explanation that has emer…
Convergence and Margin of Adversarial Training on Separable Data
Zachary Charles, Shashank Rajput, Stephen Wright +1
Adversarial training is a technique for training robust machine learning models. To encourage robustness, it iteratively computes adversarial examples for the model, and then re-tr…
Does Data Augmentation Lead to Positive Margin?
Shashank Rajput, Zhili Feng, Zachary Charles +2
Data augmentation (DA) is commonly used during model training, as it significantly improves test error and model robustness. DA artificially expands the training set by applying ra…
MLSys: The New Frontier of Machine Learning Systems
Alexander Ratner, Dan Alistarh, Gustavo Alonso +66
Machine learning (ML) techniques are enjoying rapidly increasing adoption. However, designing and implementing the systems that support ML models in real-world deployments remains…
ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Coding
Hongyi Wang, Zachary Charles, Dimitris Papailiopoulos
We present ErasureHead, a new approach for distributed gradient descent (GD) that mitigates system delays by employing approximate gradient coding. Gradient coded distributed GD us…