Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning

Seanie Lee, Hae Beom Lee, Juho Lee +1

Multilingual models jointly pretrained on multiple languages have achieved remarkable performance on various multilingual downstream tasks. Moreover, models finetuned on a single m…