m2mKD: Module-to-Module Knowledge Distillation for Modular Transformers

Ka Man Lo, Yiming Liang, Wenyu Du +5

Modular neural architectures are gaining attention for their powerful generalization and efficient adaptation to new domains. However, training these models poses challenges due to…