LoTR: Low Tensor Rank Weight Adaptation

Daniel Bershatsky, Daria Cherniuk, Talgat Daulbaev +2

In this paper we generalize and extend an idea of low-rank adaptation (LoRA) of large language models (LLMs) based on Transformer architecture. Widely used LoRA-like methods of fin…