On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference

Daniel Dold, Emanuel Sommer, Julius Kobialka +2

While parameter-efficient fine-tuning methods like low-rank adaptation (LoRA) are standard for large language models, principled estimation of epistemic uncertainty remains challen…