LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters

Klaudia Bałazy, Mohammadreza Banaei, Karl Aberer +1

The growth of large language models underscores the need for parameter-efficient fine-tuning. Despite its popularity, LoRA encounters storage and computational challenges when depl…