SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task
arXiv:2010.05122
Abstract
In this paper, we introduced our joint team SJTU-NICT 's participation in the WMT 2020 machine translation shared task. In this shared task, we participated in four translation directions of three language pairs: English-Chinese, English-Polish on supervised machine translation track, German-Upper Sorbian on low-resource and unsupervised machine translation tracks. Based on different conditions of language pairs, we have experimented with diverse neural machine translation (NMT) techniques: document-enhanced NMT, XLM pre-trained language model enhanced NMT, bidirectional translation as a pre-training, reference language based UNMT, data-dependent gaussian prior objective, and BT-BLEU collaborative filtering self-training. We also used the TF-IDF algorithm to filter the training set to obtain a domain more similar set with the test set for finetuning. In our submissions, the primary systems won the first place on English to Chinese, Polish to English, and German to Upper Sorbian translation directions.
arXiv commentWMT20
References in corpus (15)
- Longformer: The Long-Document Transformer
- Dual Learning for Machine Translation
- Unsupervised Statistical Machine Translation
- Reformer: The Efficient Transformer
- Incorporating BERT into Neural Machine Translation
- fairseq: A Fast, Extensible Toolkit for Sequence Modeling
- Revisiting Self-Training for Neural Sequence Generation
- PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
- Modeling Multi-turn Conversation with Deep Utterance Aggregation
- Context-Aware Learning for Neural Machine Translation
- A Survey on Document-level Neural Machine Translation: Methods and Evaluation
- Improving Conditioning in Context-Aware Sequence to Sequence Models
- Reference Language based Unsupervised Neural Machine Translation
- Explicit Sentence Compression for Neural Machine Translation
- Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios