Ladder Up, Memory Down: Low-Cost Fine-Tuning With Side Nets

Estelle Zheng, Nathan Cerisara, Sébastien Warichet +3

Fine-tuning large language models (LLMs) is often limited by the memory available on commodity GPUs. Parameter-efficient fine-tuning (PEFT) methods such as QLoRA reduce the number…