EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition

Hamid Nasiri, Peter Garraghan

Parameter-efficient fine-tuning methods, such as LoRA, reduces the number of trainable parameters. However, they often suffer from scalability issues and differences between their…