Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks

Rochelle Choenni, Dan Garrette, Ekaterina Shutova

Large multilingual language models typically share their parameters across all languages, which enables cross-lingual task transfer, but learning can also be hindered when training…