Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs

George Pu, Anirudh Jain, Jihan Yin +1

As foundation models continue to exponentially scale in size, efficient methods of adaptation become increasingly critical. Parameter-efficient fine-tuning (PEFT), a recent class o…