309 citations · 439 across the 15 of their papers we have counts for
18 papers · 1 filter
URA-NER: A Unified Retrieval-Augmented Framework with Retrieval Alignment and Uncertainty Reduction for Low-Resource NER
Jingyu Wang, Shijie Wu, Fusheng Jin
In-context learning (ICL) based on large language models (LLMs) has shown promising potential in alleviating performance bottlenecks caused by the limited availability of annotated…
Mitigating Entity Type Confusion in Cross-Domain NER via Multidimensional Quantification and Reasoning Enhancement
Jingyu Wang, Shijie Wu, Fusheng Jin
Cross-domain Named Entity Recognition (CD-NER) aims to transfer the rich knowledge in the source domain to the target domain. Recent studies adopting decomposition or generation pa…
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning
Genta Indra Winata, Lingjue Xie, Karthik Radhakrishnan +5
Real-life multilingual systems should be able to efficiently incorporate new languages as data distributions fed to the system evolve and shift over time. To do this, systems need…
MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies
Shiyue Zhang, Shijie Wu, Ozan Irsoy +4
Autoregressive language models are trained by minimizing the cross-entropy of the model distribution Q relative to the data distribution P -- that is, minimizing the forward cross-…
How Do Multilingual Encoders Learn Cross-lingual Representation?
Shijie Wu
NLP systems typically require support for more than one language. As different languages have different amounts of supervision, cross-lingual transfer benefits languages with littl…
Zero-shot Cross-lingual Transfer is Under-specified Optimization
Shijie Wu, Benjamin Van Durme, Mark Dredze
Pretrained multilingual encoders enable zero-shot cross-lingual transfer, but often produce unreliable models that exhibit high performance variance on the target language. We post…
Which *BERT? A Survey Organizing Contextualized Encoders
Patrick Xia, Shijie Wu, Benjamin Van Durme
Pretrained contextualized text encoders are now a staple of the NLP community. We present a survey on language representation learning with the aim of consolidating a series of sha…
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
Ekaterina Vylomova, Jennifer White, Elizabeth Salesky +25
A broad goal in natural language processing (NLP) is to develop a system that has the capacity to process any natural language. Most systems, however, are developed using data from…
The Paradigm Discovery Problem
Alexander Erdmann, Micha Elsner, Shijie Wu +2
This work treats the paradigm discovery problem (PDP), the task of learning an inflectional morphological system from unannotated sentences. We formalize the PDP and develop evalua…
Applying the Transformer to Character-level Transduction
Shijie Wu, Ryan Cotterell, Mans Hulden
The transformer has been shown to outperform recurrent neural network-based sequence-to-sequence models in various word-level NLP tasks. Yet for character-level transduction tasks,…
Are All Languages Created Equal in Multilingual BERT?
Shijie Wu, Mark Dredze
Multilingual BERT (mBERT) trained on 104 languages has shown surprisingly good cross-lingual performance on several NLP tasks, even without explicit cross-lingual signals. However,…
Emerging Cross-lingual Structure in Pretrained Language Models
Shijie Wu, Alexis Conneau, Haoran Li +2
We study the problem of multilingual masked language modeling, i.e. the training of a single model on concatenated text from multiple languages, and present a detailed study of sev…
The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection
Arya D. McCarthy, Ekaterina Vylomova, Shijie Wu +9
The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual analysis in morphology examined transfer learning of inflection between 100 language pairs, as well as cont…
Morphological Irregularity Correlates with Frequency
Shijie Wu, Ryan Cotterell, Timothy J. O'Donnell
We present a study of morphological irregularity. Following recent work, we define an information-theoretic measure of irregularity based on the predictability of forms in a langua…
Exact Hard Monotonic Attention for Character-Level Transduction
Shijie Wu, Ryan Cotterell
Many common character-level, string-to string transduction tasks, e.g., grapheme-tophoneme conversion and morphological inflection, consist almost exclusively of monotonic transduc…
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
Shijie Wu, Mark Dredze
Pretrained contextual representation models (Peters et al., 2018; Devlin et al., 2018) have pushed forward the state-of-the-art on many NLP tasks. A new release of BERT (Devlin, 20…
A Simple Joint Model for Improved Contextual Neural Lemmatization
Chaitanya Malaviya, Shijie Wu, Ryan Cotterell
English verbs have multiple forms. For instance, talk may also appear as talks, talked or talking, depending on the context. The NLP task of lemmatization seeks to map these divers…
Hard Non-Monotonic Attention for Character-Level Transduction
Shijie Wu, Pamela Shapiro, Ryan Cotterell
Character-level string-to-string transduction is an important component of various NLP tasks. The goal is to map an input string to an output string, where the strings may be of di…