works on
gradient preconditioning 1lora initialization 1low-rank adaptation 1model fine-tuning 1transfer learning 1

From the 1 of 1 linked paper with an AI index.

collaborators

1 paper

cs.LG2026

Between Gradient and Natural Gradient: A Continuum of LoRA Initializations

Dianze Liu, Farshid Ghezelbash

The paper introduces Unified LoRA (ULoRA), a two-parameter family of preconditioned gradient initializations for low‑rank adaptation, showing that optimal initialization varies by…