AROMA: Autonomous Rank-one Matrix Adaptation

Hao Nan Sheng, Zhi-yong Wang, Mingrui Yang +1

As large language models continue to grow in size, parameter-efficient fine-tuning (PEFT) has become increasingly crucial. While low-rank adaptation (LoRA) offers a solution throug…