Towards Symmetric Low-Rank Adapters

Tales Panoutsos, Rodrygo L. T. Santos, Flavio Figueiredo

In this paper, we introduce Symmetric Low-Rank Adapters, an optimized variant of LoRA with even fewer weights. This method utilizes Low-Rank Symmetric Weight Matrices to learn down…