ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning

Chenyu Zhou, Yuxuan Zeng, Qi Deng +2

Low-rank adaptation (LoRA) enables parameter-efficient fine-tuning but can suffer from redundant basis directions. Constraining the basis factor to the Stiefel manifold addresses t…