Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Improving LoRA with Variational Learning
Bai Cong, Nico Daheim, Yuesong Shen +3
Bayesian methods have recently been used to improve LoRA finetuning and, although they improve calibration, their effect on other metrics (such as accuracy) is marginal and can som…
cs.LG2024
Variational Low-Rank Adaptation Using IVON
Bai Cong, Nico Daheim, Yuesong Shen +4
We show that variational learning can significantly improve the accuracy and calibration of Low-Rank Adaptation (LoRA) without a substantial increase in the cost. We replace AdamW…
cs.LG2024
Variational Learning is Effective for Large Deep Networks
Yuesong Shen, Nico Daheim, Bai Cong +8
We give extensive empirical evidence against the common belief that variational learning is ineffective for large neural networks. We show that an optimizer called Improved Variati…