Beyond LoRA: Is Sparsity-Induced Adaptation Better?

Elijah Cadenhead, Cristian McGee, Xin Li +2

Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models. However, questions remain about the compa…