3 citations · 3 across the 4 of their papers we have counts for
5 papers
Takeuchi's Information Criteria as Generalization Measures for DNNs Close to NTK Regime
Hiroki Naganuma, Taiji Suzuki, Rio Yokota +3
Generalization measures have been studied extensively in the machine learning community to better characterize generalization gaps. However, establishing a reliable generalization…
Improving LoRA with Variational Learning
Bai Cong, Nico Daheim, Yuesong Shen +3
Bayesian methods have recently been used to improve LoRA finetuning and, although they improve calibration, their effect on other metrics (such as accuracy) is marginal and can som…
Variational Learning Finds Flatter Solutions at the Edge of Stability
Avrajit Ghosh, Bai Cong, Rio Yokota +5
Variational Learning (VL) has recently gained popularity for training deep neural networks. Part of its empirical success can be explained by theories such as PAC-Bayes bounds, min…
Variational Low-Rank Adaptation Using IVON
Bai Cong, Nico Daheim, Yuesong Shen +4
We show that variational learning can significantly improve the accuracy and calibration of Low-Rank Adaptation (LoRA) without a substantial increase in the cost. We replace AdamW…
Variational Learning is Effective for Large Deep Networks
Yuesong Shen, Nico Daheim, Bai Cong +8
We give extensive empirical evidence against the common belief that variational learning is ineffective for large neural networks. We show that an optimizer called Improved Variati…