Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer

Marko Vidoni, Ivan Vulić, Goran Glavaš

Adapter modules, additional trainable parameters that enable efficient fine-tuning of pretrained transformers, have recently been used for language specialization of multilingual t…