Latent Space Factorization in LoRA

Shashi Kumar, Yacouba Kaloga, John Mitros +2

Low-rank adaptation (LoRA) is a widely used method for parameter-efficient finetuning. However, existing LoRA variants lack mechanisms to explicitly disambiguate task-relevant info…