163 citations · 327 across the 28 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
cs.LG2020
Stochastic Optimization with Laggard Data Pipelines
Naman Agarwal, Rohan Anil, Tomer Koren +2
State-of-the-art optimization is steadily shifting towards massively parallel pipelines with extremely large batch sizes. As a consequence, CPU-bound preprocessing and disk/memory/…
cs.LG2020★ 8 cited
Disentangling Adaptive Gradient Methods from Learning Rates
Naman Agarwal, Rohan Anil, Elad Hazan +2
We investigate several confounding factors in the evaluation of optimization algorithms for deep learning. Primarily, we take a deeper look at how adaptive gradient methods interac…
cs.LG2020
No-Regret Prediction in Marginally Stable Systems
Udaya Ghai, Holden Lee, Karan Singh +2
We consider the problem of online prediction in a marginally stable linear dynamical system subject to bounded adversarial or (non-isotropic) stochastic perturbations. This poses t…