FILO: A Unified Fisher-Geometric Framework for Low-Rank Adaptation

Ziyun Liu, Fengmiao Bian, Jian-Feng Cai

Low-Rank Adaptation (LoRA) reparameterizes a weight matrix $W$ as a product of two low-rank factors, but the Jacobian $J_G$ of the generator mapping the factors to the weight matri…