SingLoRA: Low Rank Adaptation Using a Single Matrix

David Bensaïd, Noam Rotstein, Roy Velich +2

Low-Rank Adaptation (LoRA) has significantly advanced parameter-efficient fine-tuning of large pretrained models. LoRA augments the pre-trained weights of a model by adding the pro…