9 citations · 12 across the 7 of their papers we have counts for
7 papers · 1 filter
Revisiting Token Dropping Strategy in Efficient BERT Pretraining
Qihuang Zhong, Liang Ding, Juhua Liu +4
Token dropping is a recently-proposed strategy to speed up the pretraining of masked language models, such as BERT, by skipping the computation of a subset of the input tokens at s…
ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation
Zhaocong Li, Xuebo Liu, Derek F. Wong +2
Transfer learning is a simple and powerful method that can be used to boost model performance of low-resource neural machine translation (NMT). Existing transfer learning methods f…
Improving Simultaneous Machine Translation with Monolingual Data
Hexuan Deng, Liang Ding, Xuebo Liu +3
Simultaneous machine translation (SiMT) is usually done via sequence-level knowledge distillation (Seq-KD) from a full-sentence neural machine translation (NMT) model. However, the…
Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling
Zhijun Wang, Xuebo Liu, Min Zhang
Existing research generally treats Chinese character as a minimum unit for representation. However, such Chinese character representation will suffer two bottlenecks: 1) Learning b…
Revisiting Grammatical Error Correction Evaluation and Beyond
Peiyuan Gong, Xuebo Liu, Heyan Huang +1
Pretraining-based (PT-based) automatic evaluation metrics (e.g., BERTScore and BARTScore) have been widely used in several sentence generation tasks (e.g., machine translation and…
BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representation
Zheng Zhang, Liang Ding, Dazhao Cheng +3
Data augmentations (DA) are the cores to achieving robust sequence-to-sequence learning on various natural language processing (NLP) tasks. However, most of the DA approaches force…