RoCoFT: Efficient Finetuning of Large Language Models with Row-Column Updates

Md Kowsher, Tara Esmaeilbeig, Chun-Nam Yu +3

We propose RoCoFT, a parameter-efficient fine-tuning method for large-scale language models (LMs) based on updating only a few rows and columns of the weight matrices in transforme…