Bayesian Fine-tuning in Projected Subspaces

Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1

Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning of large models by decomposing weight updates into low-rank matrices, significantly reducing storage and computat…