Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models

Emre Onal, Klemens Flöge, Emma Caldwell +2

Fine-tuned Large Language Models (LLMs) often suffer from overconfidence and poor calibration, particularly when fine-tuned on small datasets. To address these challenges, we propo…