11 citations · 18 across the 3 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2019★ 11 cited
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise
Yeming Wen, Kevin Luk, Maxime Gazeau +3
The choice of batch-size in a stochastic optimization algorithm plays a substantial role for both optimization and generalization. Increasing the batch-size used typically improves…
cs.LG2018★ 7 cited
A Coordinate-Free Construction of Scalable Natural Gradient
Kevin Luk, Roger Grosse
Most neural networks are trained using first-order optimization methods, which are sensitive to the parameterization of the model. Natural gradient descent is invariant to smooth r…