LoRTA: Low Rank Tensor Adaptation of Large Language Models

Ignacio Hounie, Charilaos Kanatsoulis, Arnuv Tandon +1

Low Rank Adaptation (LoRA) is a popular Parameter Efficient Fine Tuning (PEFT) method that effectively adapts large pre-trained models for downstream tasks. LoRA parameterizes mode…