56 citations · 140 across the 23 of their papers we have counts for
23 papers
Uncertainty-Penalized Direct Preference Optimization
Sam Houliston, Alizée Pace, Alexander Immer +1
Aligning Large Language Models (LLMs) to human preferences in content, style, and presentation is challenging, in part because preferences are varied, context-dependent, and someti…
Influence Functions for Scalable Data Attribution in Diffusion Models
Bruno Mlodozeniec, Runa Eschenhagen, Juhan Bae +3
Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In th…
Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihood
Rayen Dhahri, Alexander Immer, Betrand Charpentier +2
Neural network sparsification is a promising avenue to save computational time and memory costs, especially in an age where many successful AI models are becoming too large to naïv…
Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI
Theodore Papamarkou, Maria Skoularidou, Konstantina Palla +22
In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language dat…
Uncertainty in Graph Contrastive Learning with Bayesian Neural Networks
Alexander Möllers, Alexander Immer, Elvin Isufi +1
Graph contrastive learning has shown great promise when labeled data is scarce, but large unlabeled datasets are available. However, it often does not take uncertainty estimation i…
Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
Runa Eschenhagen, Alexander Immer, Richard E. Turner +2
The core components of many modern neural network architectures, such as transformers, convolutional, or graph neural networks, can be expressed as linear layers with $\textit{weig…
Learning Layer-wise Equivariances Automatically using Gradients
Tycho F. A. van der Ouderaa, Alexander Immer, Mark van der Wilk
Convolutions encode equivariance symmetries into neural networks leading to better generalisation performance. However, symmetries provide fixed hard constraints on the functions a…
Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion
Alexandru Meterez, Amir Joudaki, Francesco Orabona +3
Normalization layers are one of the key building blocks for deep neural networks. Several theoretical studies have shown that batch normalization improves the signal propagation, b…
Hodge-Aware Contrastive Learning
Alexander Möllers, Alexander Immer, Vincent Fortuin +1
Simplicial complexes prove effective in modeling data with multiway dependencies, such as data defined along the edges of networks or within other higher-order structures. Their sp…
Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels
Alexander Immer, Tycho F. A. van der Ouderaa, Mark van der Wilk +2
Selecting hyperparameters in deep learning greatly impacts its effectiveness but requires manual effort and expertise. Recent works show that Bayesian model selection with Laplace…
Improving Neural Additive Models with Bayesian Principles
Kouroche Bouchiat, Alexander Immer, Hugo Yèche +2
Neural additive models (NAMs) enhance the transparency of deep neural networks by handling input features in separate additive sub-networks. However, they lack inherent mechanisms…
Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization
Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen +1
The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks. It is theoretically compelling since it can be see…
On the Identifiability and Estimation of Causal Location-Scale Noise Models
Alexander Immer, Christoph Schultheiss, Julia E. Vogt +3
We study the class of location-scale or heteroscedastic noise models (LSNMs), in which the effect can be written as a function of the cause and a noise source independe…
Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations
Alexander Immer, Tycho F. A. van der Ouderaa, Gunnar Rätsch +2
Data augmentation is commonly applied to improve performance of deep learning by enforcing the knowledge that certain transformations on the input preserve the output. Currently, t…
Pathologies in priors and inference for Bayesian transformers
Tristan Cinquin, Alexander Immer, Max Horn +1
In recent years, the transformer has established itself as a workhorse in many applications ranging from natural language processing to reinforcement learning. Similarly, Bayesian…
Probing as Quantifying Inductive Bias
Alexander Immer, Lucas Torroba Hennigen, Vincent Fortuin +1
Pre-trained contextual representations have led to dramatic performance improvements on a range of downstream tasks. Such performance improvements have motivated researchers to qua…
Laplace Redux -- Effortless Bayesian Deep Learning
Erik Daxberger, Agustinus Kristiadi, Alexander Immer +3
Bayesian formulations of deep learning have been shown to have compelling theoretical properties and offer practical functional benefits, such as improved predictive uncertainty qu…
Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning
Alexander Immer, Matthias Bauer, Vincent Fortuin +2
Marginal-likelihood based model-selection, even though promising, is rarely used in deep learning due to estimation difficulties. Instead, most approaches rely on validation data,…
Improving predictions of Bayesian neural nets via local linearization
Alexander Immer, Maciej Korzepa, Matthias Bauer
The generalized Gauss-Newton (GGN) approximation is often used to make practical Bayesian deep learning approaches scalable by replacing a second order derivative with a product of…
Disentangling the Gauss-Newton Method and Approximate Inference for Neural Networks
Alexander Immer
In this thesis, we disentangle the generalized Gauss-Newton and approximate inference for Bayesian deep learning. The generalized Gauss-Newton method is an optimization method that…
Continual Deep Learning by Functional Regularisation of Memorable Past
Pingbo Pan, Siddharth Swaroop, Alexander Immer +3
Continually learning new skills is important for intelligent systems, yet standard deep learning methods suffer from catastrophic forgetting of the past. Recent works address this…
Variational Inference with Numerical Derivatives: variance reduction through coupling
Alexander Immer, Guillaume P. Dehaene
The Black Box Variational Inference (Ranganath et al. (2014)) algorithm provides a universal method for Variational Inference, but taking advantage of special properties of the app…
Approximate Inference Turns Deep Networks into Gaussian Processes
Mohammad Emtiyaz Khan, Alexander Immer, Ehsan Abedi +1
Deep neural networks (DNN) and Gaussian processes (GP) are two powerful models with several theoretical connections relating them, but the relationship between their training metho…