Is Modularity Transferable? A Case Study through the Lens of Knowledge Distillation

Mateusz Klimaszewski, Piotr Andruszkiewicz, Alexandra Birch

The rise of Modular Deep Learning showcases its potential in various Natural Language Processing applications. Parameter-efficient fine-tuning (PEFT) modularity has been shown to w…