10 papers
Variational Model Merging for Pareto Front Estimation in Multitask Finetuning
Hugo Monzón Maldonado, Nico Daheim, Thomas Möllenhoff +2
Pareto fronts are useful to find good task-mixing strategies for multitask finetuning, but they are also costly to compute. To reduce costs, recent works have used existing model m…
Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behavior
Christopher J. Anders, Henrique Da Silva Gameiro, Nico Daheim +1
One way to understand LLM behavior is to trace its output back to the training data. Two types of measures are commonly used for output tracing: data-similarity and data-influence.…
SVRG and Beyond via Posterior Correction
Nico Daheim, Thomas Möllenhoff, Ming Liang Ang +1
Stochastic Variance Reduced Gradient (SVRG) and its variants aim to speed-up training by using gradient corrections. Originally proposed over a decade ago, these methods have never…
Joint Model and Data Sparsification via the Marginal Likelihood
Alexander Timans, Thomas Möllenhoff, Christian A. Naesseth +2
Sparse recovery in linear systems underpins applications from signal processing to high-dimensional regression. Sparse Bayesian Learning, grounded in the principle of automatic rel…
Position: agentic AI orchestration should be Bayes-consistent
Theodore Papamarkou, Pierre Alquier, Matthias Bauer +27
LLMs excel at predictive tasks and complex reasoning tasks, but many high-value deployments rely on decisions under uncertainty, for example, which tool to call, which expert to co…
Compact Memory for Continual Logistic Regression
Yohan Jung, Hyungi Lee, Wenlong Chen +4
Despite recent progress, continual learning still does not match the performance of batch training. To avoid catastrophic forgetting, we need to build compact memory of essential p…