DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models

Xiaolin Hu, Xiang Cheng, Peiyu Liu +4

Low-rank adaptation (LoRA) reduces the computational and memory demands of fine-tuning large language models (LLMs) by approximating updates with low-rank matrices. However, low-ra…