AdapterDrop: On the Efficiency of Adapters in Transformers

Andreas Rücklé, Gregor Geigle, Max Glockner +4

Massively pre-trained transformer models are computationally expensive to fine-tune, slow for inference, and have large storage requirements. Recent approaches tackle these shortco…