CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA

Gengyu Zhang, Haiyin Ran, Zhengbao He +4

As the scale of large pre-trained models continues to grow, fine-tuning them under limited memory budgets has become increasingly challenging. Low-Rank Adaptation (LoRA), currently…