4 papers · 1 filter
Closed-Form Last Layer Optimization
Alexandre Galashov, Nathaël Da Costa, Liyuan Xu +2
Neural networks are typically optimized with variants of stochastic gradient descent. Under a squared loss, however, the optimal solution to the linear last layer weights is known…
Rethinking Approximate Gaussian Inference in Classification
Bálint Mucsányi, Nathaël Da Costa, Philipp Hennig
In classification tasks, softmax functions are ubiquitously used as output activations to produce predictive probabilities. Such outputs only capture aleatoric uncertainty. To capt…
laplax -- Laplace Approximations with JAX
Tobias Weber, Bálint Mucsányi, Lenard Rommel +4
The Laplace approximation provides a scalable and efficient means of quantifying weight-space uncertainty in deep neural networks, enabling the application of Bayesian tools such a…
Debiasing Mini-Batch Quadratics for Applications in Deep Learning
Lukas Tatzel, Bálint Mucsányi, Osane Hackel +1
Quadratic approximations form a fundamental building block of machine learning methods. E.g., second-order optimizers try to find the Newton step into the minimum of a local quadra…