4 papers
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise
Yeming Wen, Kevin Luk, Maxime Gazeau +3
The choice of batch-size in a stochastic optimization algorithm plays a substantial role for both optimization and generalization. Increasing the batch-size used typically improves…
A Coordinate-Free Construction of Scalable Natural Gradient
Kevin Luk, Roger Grosse
Most neural networks are trained using first-order optimization methods, which are sensitive to the parameterization of the model. Natural gradient descent is invariant to smooth r…
Scalable Recommender Systems through Recursive Evidence Chains
Elias Tragas, Calvin Luo, Maxime Gazeau +2
Recommender systems can be formulated as a matrix completion problem, predicting ratings from user and item parameter vectors. Optimizing these parameters by subsampling data becom…
Log Picard algebroids and meromorphic line bundles
Marco Gualtieri, Kevin Luk
We introduce logarithmic Picard algebroids, a natural class of Lie algebroids adapted to a simple normal crossings divisor on a smooth projective variety. We show that such algebro…