An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning

Mohammad Akbar-Tajari, Sara Rajaee, Mohammad Taher Pilehvar

Parameter-efficient fine-tuning approaches have recently garnered a lot of attention. Having considerably lower number of trainable weights, these methods can bring about scalabili…