Fast Forwarding Low-Rank Training

Adir Rahamim, Naomi Saphra, Sara Kangaslahti +1

Parameter efficient finetuning methods like low-rank adaptation (LoRA) aim to reduce the computational costs of finetuning pretrained Language Models (LMs). Enabled by these low-ra…