6 papers
On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
Daniel Dold, Emanuel Sommer, Julius Kobialka +2
While parameter-efficient fine-tuning methods like low-rank adaptation (LoRA) are standard for large language models, principled estimation of epistemic uncertainty remains challen…
On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors
Julius Kobialka, Emanuel Sommer, Chris Kolb +3
Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space…
Paths and Ambient Spaces in Neural Loss Landscapes
Daniel Dold, Julius Kobialka, Nicolai Palm +3
Understanding the structure of neural network loss surfaces, particularly the emergence of low-loss tunnels, is critical for advancing neural network theory and practice. In this p…
How Inverse Conditional Flows Can Serve as a Substitute for Distributional Regression
Lucas Kook, Chris Kolb, Philipp Schiele +8
Neural network representations of simple models, such as linear regression, are being studied increasingly to better understand the underlying principles of deep learning algorithm…
Bayesian Semi-structured Subspace Inference
Daniel Dold, David Rügamer, Beate Sick +1
Semi-structured regression models enable the joint modeling of interpretable structured and complex unstructured feature effects. The structured model part is inspired by statistic…
Transformation Models for Flexible Posteriors in Variational Bayes
Sefan Hörtling, Daniel Dold, Oliver Dürr +1
The main challenge in Bayesian models is to determine the posterior for the model parameters. Already, in models with only one or few parameters, the analytical posterior can only…