Memory-Efficient Fine-Tuning via Low-Rank Activation Compression

Jiang-Xin Shi, Wen-Da Wei, Jin-Fei Qi +3

The parameter-efficient fine-tuning paradigm has garnered significant attention with the advancement of foundation models. Although numerous methods have been proposed to reduce th…