927 citations · 2.1k across the 31 of their papers we have counts for
7 papers · 1 filter
Dual Averaging is Surprisingly Effective for Deep Learning Optimization
Samy Jelassi, Aaron Defazio
First-order stochastic optimization methods are currently the most widely used class of methods for training deep neural networks. However, the choice of the optimizer has become a…
Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization
Aaron Defazio
Momentum methods are now used pervasively within the machine learning community for training non-convex models such as deep neural networks. Empirically, they out perform tradition…
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball
Othmane Sebbouh, Robert M. Gower, Aaron Defazio
We study stochastic gradient descent (SGD) and the stochastic heavy ball method (SHB, otherwise known as the momentum method) for the general stochastic approximation problem. For…
The Power of Factorial Powers: New Parameter settings for (Stochastic) Optimization
Aaron Defazio, Robert M. Gower
The convergence rates for convex and non-convex optimization methods depend on the choice of a host of constants, including step sizes, Lyapunov function constants and momentum con…
End-to-End Variational Networks for Accelerated MRI Reconstruction
Anuroop Sriram, Jure Zbontar, Tullie Murrell +5
The slow acquisition speed of magnetic resonance imaging (MRI) has led to the development of two complementary methods: acquiring multiple views of the anatomy simultaneously (para…
Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge
Florian Knoll, Tullie Murrell, Anuroop Sriram +8
Purpose: To advance research in the field of machine learning for MR image reconstruction with an open challenge. Methods: We provided participants with a dataset of raw k-space da…