Nonlinearity as Rank: Generative Low-Rank Adapter with Radial Basis Functions

Yihao Ouyang, Shiwei Li, Haozhao Wang +6

Low-rank adaptation (LoRA) approximates the update of a pretrained weight matrix using the product of two low-rank matrices. However, standard LoRA follows an explicit-rank paradig…