AILoRA: Function-Aware Asymmetric Initialization for Low-Rank Adaptation of Large Language Models

Xiaoshuang Ji, Zhendong Zhao, Xiaoyan Gu +3

Parameter-efficient finetuning (PEFT) aims to mitigate the substantial computational and memory overhead involved in adapting large-scale pretrained models to diverse downstream ta…