BaLoRA: Bayesian Low-Rank Adaptation of Large Scale Models

Dario Coscia, Sindy Löwe, Max Welling

Low-Rank Adaptation (LoRA) has become the standard for fine-tuning large pre-trained models at reduced computational cost. However, its low-rank point-estimate updates limit expres…