34 citations · 108 across the 30 of their papers we have counts for
5 papers · 1 filter
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
Wu Lin, Felix Dangel, Runa Eschenhagen +4
Second-order methods such as KFAC can be useful for neural net training. However, they are often memory-inefficient since their preconditioning Kronecker factors are dense, and num…
Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks
Ahmad Rashid, Serena Hacker, Guojun Zhang +2
Discriminatively trained, deterministic neural networks are the de facto choice for classification problems. However, even though they achieve state-of-the-art results on in-domain…
On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective
Jonathan Wenger, Felix Dangel, Agustinus Kristiadi
The neural tangent kernel (NTK) has garnered significant attention as a theoretical framework for describing the behavior of large-scale neural networks. Kernel methods are theoret…
Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization
Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen +1
The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks. It is theoretically compelling since it can be see…
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization
Agustinus Kristiadi, Felix Dangel, Philipp Hennig
Model reparametrization, which follows the change-of-variable rule of calculus, is a popular way to improve the training of neural nets. But it can also be problematic since it can…