Efficient Storage of Fine-Tuned Models via Low-Rank Approximation of Weight Residuals

Simo Ryu, Seunghyun Seo, Jaejun Yoo

In this paper, we present an efficient method for storing fine-tuned models by leveraging the low-rank properties of weight residuals. Our key observation is that weight residuals…