105 citations · 147 across the 17 of their papers we have counts for
19 papers
MegaWika: Millions of reports and their sources across 50 diverse languages
Samuel Barham, Orion Weller, Michelle Yuan +9
To foster the development of new models for collaborative AI-assisted report generation, we introduce MegaWika, consisting of 13 million Wikipedia articles in 50 diverse languages,…
Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution
Tianjian Li, Kenton Murray
Zero-shot cross-lingual transfer is when a multilingual model is trained to perform a task in one language and then is applied to another language. Although the zero-shot cross-lin…
Exploring Representational Disparities Between Multilingual and Bilingual Translation Models
Neha Verma, Kenton Murray, Kevin Duh
Multilingual machine translation has proven immensely useful for both parameter efficiency and overall performance across many language pairs via complete multilingual parameter sh…
Condensing Multilingual Knowledge with Lightweight Language-Specific Modules
Haoran Xu, Weiting Tan, Shuyue Stella Li +4
Incorporating language-specific (LS) modules is a proven method to boost performance in multilingual machine translation. This approach bears similarity to Mixture-of-Experts (MoE)…
Towards Being Parameter-Efficient: A Stratified Sparsely Activated Transformer with Dynamic Capacity
Haoran Xu, Maha Elbayad, Kenton Murray +2
Mixture-of-experts (MoE) models that employ sparse activation have demonstrated effectiveness in significantly increasing the number of parameters while maintaining low computation…
Language Agnostic Code-Mixing Data Augmentation by Predicting Linguistic Patterns
Shuyue Stella Li, Kenton Murray
In this work, we focus on intrasentential code-mixing and propose several different Synthetic Code-Mixing (SCM) data augmentation methods that outperform the baseline on downstream…