3 citations · 3 across the 6 of their papers we have counts for
6 papers
SOAP-Bubbles: Structured Weight Uncertainty for Neural Networks
Adrian Robert Minut, Nico Daheim, Marco Miani +3
Structured weight-uncertainty can improve many aspects of deep learning, but it remains costly to estimate and difficult to implement. Here, we show that these issues can be addres…
Fast and Slow Variational Continual Learning
Subarnaduti Paul, Yohan Jung, Mohammad Emtiyaz Khan +3
Continual learning remains a major challenge for modern deep networks, partly because commonly used optimizers lack inherent mechanisms for continual adaptation. One such natural m…
Federated ADMM from Bayesian Duality
Thomas Möllenhoff, Siddharth Swaroop, Finale Doshi-Velez +1
We propose a new Bayesian approach to generalize the federated Alternating Direction Method of Multipliers (ADMM). We show that the solutions of variational-Bayesian (VB) objective…
Connecting Federated ADMM to Bayes
Siddharth Swaroop, Mohammad Emtiyaz Khan, Finale Doshi-Velez
We provide new connections between two distinct federated learning approaches based on (i) ADMM and (ii) Variational Bayes (VB), and propose new variants by combining their complem…
Variational Low-Rank Adaptation Using IVON
Bai Cong, Nico Daheim, Yuesong Shen +4
We show that variational learning can significantly improve the accuracy and calibration of Low-Rank Adaptation (LoRA) without a substantial increase in the cost. We replace AdamW…
Variational Learning is Effective for Large Deep Networks
Yuesong Shen, Nico Daheim, Bai Cong +8
We give extensive empirical evidence against the common belief that variational learning is ineffective for large neural networks. We show that an optimizer called Improved Variati…