AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks

Chin-Lun Fu, Zih-Ching Chen, Yun-Ru Lee +1

Transformer-based pre-trained models with millions of parameters require large storage. Recent approaches tackle this shortcoming by training adapters, but these approaches still r…