Run LoRA Run: Faster and Lighter LoRA Implementations

Daria Cherniuk, Aleksandr Mikhalev, Ivan Oseledets

LoRA is a technique that reduces the number of trainable parameters in a neural network by introducing low-rank adapters to linear layers. This technique is used both for fine-tuni…