LoRA ensembles for large language model fine-tuning

Xi Wang, Laurence Aitchison, Maja Rudolph

Finetuned LLMs often exhibit poor uncertainty quantification, manifesting as overconfidence, poor calibration, and unreliable prediction results on test data or out-of-distribution…