2 citations · 3 across the 12 of their papers we have counts for
15 papers
Train for Many, Update with One: Improving Multi-Sample Reasoning via Fenchel-Bregman Posterior Learning
Sungjun Lim, Adrian Robert Minut, Nico Carlsen Daheim +3
Multi-sample reasoning measures candidate coverage, the probability that at least one of candidates is correct. However, minimizing the average loss of individual attempts does…
A Generalization of Amari's Bayesian Duality
Mohammad Emtiyaz Khan, Thomas Möllenhoff
Amari's contributions to information geometry and machine learning are well known. Here, we revisit Amari's work on Bayesian duality which has not received as much attention. We co…
Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behavior
Christopher J. Anders, Henrique Da Silva Gameiro, Nico Daheim +1
One way to understand LLM behavior is to trace its output back to the training data. Two types of measures are commonly used for output tracing: data-similarity and data-influence.…
Joint Model and Data Sparsification via the Marginal Likelihood
Alexander Timans, Thomas Möllenhoff, Christian A. Naesseth +2
Sparse recovery in linear systems underpins applications from signal processing to high-dimensional regression. Sparse Bayesian Learning, grounded in the principle of automatic rel…
Position: agentic AI orchestration should be Bayes-consistent
Theodore Papamarkou, Pierre Alquier, Matthias Bauer +27
LLMs excel at predictive tasks and complex reasoning tasks, but many high-value deployments rely on decisions under uncertainty, for example, which tool to call, which expert to co…
SVRG and Beyond via Posterior Correction
Nico Daheim, Thomas Möllenhoff, Ming Liang Ang +1
Stochastic Variance Reduced Gradient (SVRG) and its variants aim to speed-up training by using gradient corrections. Originally proposed over a decade ago, these methods have never…
Compact Memory for Continual Logistic Regression
Yohan Jung, Hyungi Lee, Wenlong Chen +4
Despite recent progress, continual learning still does not match the performance of batch training. To avoid catastrophic forgetting, we need to build compact memory of essential p…
Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
Navish Kumar, Thomas Möllenhoff, Mohammad Emtiyaz Khan +1
Variational inference with natural-gradient descent often shows fast convergence in practice, but its theoretical convergence guarantees have been challenging to establish. This is…
Log-Normal Multiplicative Dynamics for Stable Low-Precision Training of Large Networks
Keigo Nishida, Eren Mehmet Kıral, Kenichi Bannai +2
Studies in neuroscience have shown that biological synapses follow a log-normal distribution whose transitioning can be explained by noisy multiplicative dynamics. Biological netwo…
Improving LoRA with Variational Learning
Bai Cong, Nico Daheim, Yuesong Shen +3
Bayesian methods have recently been used to improve LoRA finetuning and, although they improve calibration, their effect on other metrics (such as accuracy) is marginal and can som…
Variational Learning Finds Flatter Solutions at the Edge of Stability
Avrajit Ghosh, Bai Cong, Rio Yokota +5
Variational Learning (VL) has recently gained popularity for training deep neural networks. Part of its empirical success can be explained by theories such as PAC-Bayes bounds, min…
Variational Model Merging for Pareto Front Estimation in Multitask Finetuning
Hugo Monzón Maldonado, Nico Daheim, Thomas Möllenhoff +2
Pareto fronts are useful to find good task-mixing strategies for multitask finetuning, but they are also costly to compute. To reduce costs, recent works have used existing model m…
Conformal Prediction via Regression-as-Classification
Etash Guha, Shlok Natarajan, Thomas Möllenhoff +2
Conformal prediction (CP) for regression can be challenging, especially when the output distribution is heteroscedastic, multimodal, or skewed. Some of the issues can be addressed…
The Memory Perturbation Equation: Understanding Model's Sensitivity to Data
Peter Nickl, Lu Xu, Dharmesh Tailor +2
Understanding model's sensitivity to its training data is crucial but can also be challenging and costly, especially during training. To simplify such issues, we present the Memory…
Model Merging by Uncertainty-Based Gradient Matching
Nico Daheim, Thomas Möllenhoff, Edoardo Maria Ponti +2
Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail? Here, we connect the inaccuracy of weight…