Understanding and Guiding Layer Placement in Parameter-Efficient Fine-Tuning of Large Language Models

Yichen Xu, Yuyang Liang, Shan Dai +3

As large language models (LLMs) continue to grow, the cost of full-parameter fine-tuning has made parameter-efficient fine-tuning (PEFT) the default strategy for downstream adaptat…