425 citations · 1.1k across the 32 of their papers we have counts for
Showing 2019 · cs.LGShow all
3 papers · 2 filters
cs.LG2019★ 1 cited
A Dynamic Sampling Adaptive-SGD Method for Machine Learning
Achraf Bahamou, Donald Goldfarb
We propose a stochastic optimization method for minimizing loss functions, expressed as an expected value, that adaptively controls the batch size used in the computation of gradie…
cs.LG2019★ 12 cited
Efficient Subsampled Gauss-Newton and Natural Gradient Methods for Training Neural Networks
Yi Ren, Donald Goldfarb
We present practical Levenberg-Marquardt variants of Gauss-Newton and natural gradient methods for solving non-convex optimization problems that arise in training deep neural netwo…
cs.LG2019★ 1 cited
Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models: Extension
Yunfei Teng, Wenbo Gao, Francois Chalus +3
We consider distributed optimization under communication constraints for training deep learning models. We propose a new algorithm, whose parameter updates rely on two forces: a re…