59 citations · 186 across the 12 of their papers we have counts for
12 papers
SOAP-Bubbles: Structured Weight Uncertainty for Neural Networks
Adrian Robert Minut, Nico Daheim, Marco Miani +3
Structured weight-uncertainty can improve many aspects of deep learning, but it remains costly to estimate and difficult to implement. Here, we show that these issues can be addres…
Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
Wu Lin, Valentin Duruisseaux, Melvin Leok +3
Riemannian submanifold optimization with momentum is computationally challenging because, to ensure that the iterates remain on the submanifold, we often need to solve difficult di…
Structured second-order methods via natural gradient descent
Wu Lin, Frank Nielsen, Mohammad Emtiyaz Khan +1
In this paper, we propose new structured second-order methods and structured adaptive-gradient methods obtained by performing natural-gradient descent on structured parameter space…
Tractable structured natural gradient descent using local parameterizations
Wu Lin, Frank Nielsen, Mohammad Emtiyaz Khan +1
Natural-gradient descent (NGD) on structured parameter spaces (e.g., low-rank covariances) is computationally challenging due to difficult Fisher-matrix computations. We address th…
Handling the Positive-Definite Constraint in the Bayesian Learning Rule
Wu Lin, Mark Schmidt, Mohammad Emtiyaz Khan
The Bayesian learning rule is a natural-gradient variational inference method, which not only contains many existing learning algorithms as special cases but also enables the desig…
Stein's Lemma for the Reparameterization Trick with Exponential Family Mixtures
Wu Lin, Mohammad Emtiyaz Khan, Mark Schmidt
Stein's method (Stein, 1973; 1981) is a powerful tool for statistical applications and has significantly impacted machine learning. Stein's lemma plays an essential role in Stein's…