Sharp Generalization Bounds for Foundation Models with Asymmetric Randomized Low-Rank Adapters

Anastasis Kratsios, Tin Sum Cheng, Aurelien Lucchi +1

Low-Rank Adaptation (LoRA) has emerged as a widely adopted parameter-efficient fine-tuning (PEFT) technique for foundation models. Recent work has highlighted an inherent asymmetry…