124 citations · 528 across the 41 of their papers we have counts for
23 papers · 1 filter
Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
Wu Lin, Valentin Duruisseaux, Melvin Leok +3
Riemannian submanifold optimization with momentum is computationally challenging because, to ensure that the iterates remain on the submanifold, we often need to solve difficult di…
Subset-of-Data Variational Inference for Deep Gaussian-Processes Regression
Ayush Jain, P. K. Srijith, Mohammad Emtiyaz Khan
Deep Gaussian Processes (DGPs) are multi-layer, flexible extensions of Gaussian processes but their training remains challenging. Sparse approximations simplify the training but of…
Structured second-order methods via natural gradient descent
Wu Lin, Frank Nielsen, Mohammad Emtiyaz Khan +1
In this paper, we propose new structured second-order methods and structured adaptive-gradient methods obtained by performing natural-gradient descent on structured parameter space…
The Bayesian Learning Rule
Mohammad Emtiyaz Khan, Håvard Rue
We show that many machine-learning algorithms are specific instances of a single algorithm called the \emph{Bayesian learning rule}. The rule, derived from Bayesian principles, yie…
Bridging the Gap Between Target Networks and Functional Regularization
Alexandre Piché, Valentin Thomas, Rafael Pardinas +4
Bootstrapping is behind much of the successes of deep Reinforcement Learning. However, learning the value function via bootstrapping often leads to unstable training due to fast-ch…
Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning
Alexander Immer, Matthias Bauer, Vincent Fortuin +2
Marginal-likelihood based model-selection, even though promising, is rarely used in deep learning due to estimation difficulties. Instead, most approaches rely on validation data,…