Learning Adapter Rank via Symmetry Breaking

Cooper Doyle, Andy Hu, Rebecca Chan +1

Low-rank adaptation is effective partly because downstream updates lie in a low-dimensional subspace, but the latent rank coordinates of LoRA are not identifiable: any invertible r…