8 citations · 24 across the 28 of their papers we have counts for
7 papers · 1 filter
Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
Navish Kumar, Thomas Möllenhoff, Mohammad Emtiyaz Khan +1
Variational inference with natural-gradient descent often shows fast convergence in practice, but its theoretical convergence guarantees have been challenging to establish. This is…
Log-Normal Multiplicative Dynamics for Stable Low-Precision Training of Large Networks
Keigo Nishida, Eren Mehmet Kıral, Kenichi Bannai +2
Studies in neuroscience have shown that biological synapses follow a log-normal distribution whose transitioning can be explained by noisy multiplicative dynamics. Biological netwo…
Improving LoRA with Variational Learning
Bai Cong, Nico Daheim, Yuesong Shen +3
Bayesian methods have recently been used to improve LoRA finetuning and, although they improve calibration, their effect on other metrics (such as accuracy) is marginal and can som…
Federated ADMM from Bayesian Duality
Thomas Möllenhoff, Siddharth Swaroop, Finale Doshi-Velez +1
We propose a new Bayesian approach to generalize the federated Alternating Direction Method of Multipliers (ADMM). We show that the solutions of variational-Bayesian (VB) objective…
Variational Learning Finds Flatter Solutions at the Edge of Stability
Avrajit Ghosh, Bai Cong, Rio Yokota +5
Variational Learning (VL) has recently gained popularity for training deep neural networks. Part of its empirical success can be explained by theories such as PAC-Bayes bounds, min…
Uncertainty-Aware Decoding with Minimum Bayes Risk
Nico Daheim, Clara Meister, Thomas Möllenhoff +1
Despite their outstanding performance in the majority of scenarios, contemporary language models still occasionally generate undesirable outputs, for example, hallucinated text. Wh…
Natural Variational Annealing for Multimodal Optimization
Tâm LeMinh, Julyan Arbel, Thomas Möllenhoff +2
We introduce a new multimodal optimization approach called Natural Variational Annealing (NVA) that combines the strengths of three foundational concepts to simultaneously search f…