Hypernetwork-Driven Low-Rank Adaptation Across Attention Heads

Nghiem T. Diep, Dung Le, Tuan Truong +3

Parameter-efficient fine-tuning (PEFT) has emerged as a powerful paradigm for adapting large-scale pre-trained models to downstream tasks with minimal additional parameters. Among…