Faster Than SVD, Smarter Than SGD: The OPLoRA Alternating Update

Abdulla Jasem Almansoori, Maria Ivanova, Andrey Veprikov +3

Low-Rank Adaptation (LoRA) fine-tunes large models by learning low-rank updates on top of frozen weights, dramatically reducing trainable parameters and memory. However, there is s…