3 citations · 3 across the 4 of their papers we have counts for
4 papers · 1 filter
Takeuchi's Information Criteria as Generalization Measures for DNNs Close to NTK Regime
Hiroki Naganuma, Taiji Suzuki, Rio Yokota +3
Generalization measures have been studied extensively in the machine learning community to better characterize generalization gaps. However, establishing a reliable generalization…
Improving LoRA with Variational Learning
Bai Cong, Nico Daheim, Yuesong Shen +3
Bayesian methods have recently been used to improve LoRA finetuning and, although they improve calibration, their effect on other metrics (such as accuracy) is marginal and can som…
Variational Low-Rank Adaptation Using IVON
Bai Cong, Nico Daheim, Yuesong Shen +4
We show that variational learning can significantly improve the accuracy and calibration of Low-Rank Adaptation (LoRA) without a substantial increase in the cost. We replace AdamW…
Variational Learning is Effective for Large Deep Networks
Yuesong Shen, Nico Daheim, Bai Cong +8
We give extensive empirical evidence against the common belief that variational learning is ineffective for large neural networks. We show that an optimizer called Improved Variati…