Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs

Yifei Zhang, Hao Zhu, Aiwei Liu +3

Fine-tuning Large Language Models (LLMs) has become a crucial technique for adapting pre-trained models to downstream tasks. However, the enormous size of LLMs poses significant ch…