309 citations · 439 across the 15 of their papers we have counts for
6 papers · 1 filter
Emerging Cross-lingual Structure in Pretrained Language Models
Shijie Wu, Alexis Conneau, Haoran Li +2
We study the problem of multilingual masked language modeling, i.e. the training of a single model on concatenated text from multiple languages, and present a detailed study of sev…
The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection
Arya D. McCarthy, Ekaterina Vylomova, Shijie Wu +9
The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual analysis in morphology examined transfer learning of inflection between 100 language pairs, as well as cont…
Morphological Irregularity Correlates with Frequency
Shijie Wu, Ryan Cotterell, Timothy J. O'Donnell
We present a study of morphological irregularity. Following recent work, we define an information-theoretic measure of irregularity based on the predictability of forms in a langua…
Exact Hard Monotonic Attention for Character-Level Transduction
Shijie Wu, Ryan Cotterell
Many common character-level, string-to string transduction tasks, e.g., grapheme-tophoneme conversion and morphological inflection, consist almost exclusively of monotonic transduc…
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
Shijie Wu, Mark Dredze
Pretrained contextual representation models (Peters et al., 2018; Devlin et al., 2018) have pushed forward the state-of-the-art on many NLP tasks. A new release of BERT (Devlin, 20…
A Simple Joint Model for Improved Contextual Neural Lemmatization
Chaitanya Malaviya, Shijie Wu, Ryan Cotterell
English verbs have multiple forms. For instance, talk may also appear as talks, talked or talking, depending on the context. The NLP task of lemmatization seeks to map these divers…