Hydra: Multi-head Low-rank Adaptation for Parameter Efficient Fine-tuning

Sanghyeon Kim, Hyunmo Yang, Younghyun Kim +2

The recent surge in large-scale foundation models has spurred the development of efficient methods for adapting these models to various downstream tasks. Low-rank adaptation method…