S2A: A Unified Framework for Parameter and Memory Efficient Transfer Learning

Tian Jin, Enjun Du, Changwei Wang +2

Parameter-efficient transfer learning (PETL) aims to reduce the scales of pretrained models for multiple downstream tasks. However, as the models keep scaling up, the memory footpr…