147 citations · 589 across the 17 of their papers we have counts for
14 papers · 1 filter
Instruction Tuning for Large Language Models: A Survey
Shengyu Zhang, Linfeng Dong, Xiaoya Li +8
This paper surveys research works in the quickly advancing field of instruction tuning (IT), which can also be referred to as supervised fine-tuning (SFT)\footnote{In this paper, u…
Pushing the Limits of ChatGPT on NLP Tasks
Xiaofei Sun, Linfeng Dong, Xiaoya Li +8
Despite the success of ChatGPT, its performances on most NLP tasks are still well below the supervised baselines. In this work, we looked into the causes, and discovered that its s…
Text Classification via Large Language Models
Xiaofei Sun, Xiaoya Li, Jiwei Li +4
Despite the remarkable success of large-scale Language Models (LLMs) such as GPT-3, their performances still significantly underperform fine-tuned models in the task of text classi…
GPT-NER: Named Entity Recognition via Large Language Models
Shuhe Wang, Xiaofei Sun, Xiaoya Li +5
Despite the fact that large-scale Language Models (LLM) have achieved SOTA performances on a variety of NLP tasks, its performance on NER is still significantly below supervised ba…
NN-NER: Named Entity Recognition with Nearest Neighbor Search
Shuhe Wang, Xiaoya Li, Yuxian Meng +4
Inspired by recent advances in retrieval augmented methods in NLP~\citep{khandelwal2019generalization,khandelwal2020nearest,meng2021gnn}, in this paper, we introduce a nearest…
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
Zijun Sun, Xiaoya Li, Xiaofei Sun +6
Recent pretraining models in Chinese neglect two important aspects specific to the Chinese language: glyph and pinyin, which carry significant syntax and semantic information for l…
SAC: Accelerating and Structuring Self-Attention via Sparse Adaptive Connection
Xiaoya Li, Yuxian Meng, Mingxin Zhou +3
While the self-attention mechanism has been widely used in a wide variety of tasks, it has the unfortunate property of a quadratic cost with respect to the input length, which make…
LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attention
Xiaoya Li, Yuxian Meng, Arianna Yuan +2
Non-autoregressive translation (NAT) models generate multiple tokens in one forward pass and is highly efficient at inference stage compared with autoregressive translation (AT) me…
Dice Loss for Data-imbalanced NLP Tasks
Xiaoya Li, Xiaofei Sun, Yuxian Meng +4
Many NLP tasks such as tagging and machine reading comprehension are faced with the severe data imbalance issue: negative examples significantly outnumber positive examples, and th…
Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs
Yuxian Meng, Xiangyuan Ren, Zijun Sun +4
In this paper, we investigate the problem of training neural machine translation (NMT) systems with a dataset of more than 40 billion bilingual sentence pairs, which is larger than…
A Unified MRC Framework for Named Entity Recognition
Xiaoya Li, Jingrong Feng, Yuxian Meng +4
The task of named entity recognition (NER) is normally divided into nested NER and flat NER depending on whether named entities are nested or not. Models are usually separately dev…
Query-Based Named Entity Recognition
Yuxian Meng, Xiaoya Li, Zijun Sun +1
In this paper, we propose a new strategy for the task of named entity recognition (NER). We cast the task as a query-based machine reading comprehension task: e.g., the task of ext…
Is Word Segmentation Necessary for Deep Learning of Chinese Representations?
Xiaoya Li, Yuxian Meng, Xiaofei Sun +3
Segmenting a chunk of text into words is usually the first step of processing Chinese text, but its necessity has rarely been explored. In this paper, we ask the fundamental questi…
Glyce: Glyph-vectors for Chinese Character Representations
Yuxian Meng, Wei Wu, Fei Wang +8
It is intuitive that NLP tasks for logographic languages like Chinese should benefit from the use of the glyph information in those languages. However, due to the lack of rich pict…