Task-Specific Directions: Definition, Exploration, and Utilization in Parameter Efficient Fine-Tuning

Chongjie Si, Zhiyi Shi, Shifan Zhang +3

Large language models demonstrate impressive performance on downstream tasks, yet they require extensive resource consumption when fully fine-tuning all parameters. To mitigate thi…