ROSA: Random Subspace Adaptation for Efficient Fine-Tuning

Marawan Gamal Abdel Hameed, Aristides Milios, Siva Reddy +1

Model training requires significantly more memory, compared with inference. Parameter efficient fine-tuning (PEFT) methods provide a means of adapting large models to downstream ta…