Language Graph Distillation for Low-Resource Machine Translation
arXiv:1908.06258
Abstract
Neural machine translation on low-resource language is challenging due to the lack of bilingual sentence pairs. Previous works usually solve the low-resource translation problem with knowledge transfer in a multilingual setting. In this paper, we propose the concept of Language Graph and further design a novel graph distillation algorithm that boosts the accuracy of low-resource translations in the graph with forward and backward knowledge distillation. Preliminary experiments on the TED talks multilingual dataset demonstrate the effectiveness of our proposed method. Specifically, we improve the low-resource translation pair by more than 3.13 points in terms of BLEU score.
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- Sequence to Sequence Learning with Neural Networks
- Effective Approaches to Attention-based Neural Machine Translation
- Dual Learning for Machine Translation
- MASS: Masked Sequence to Sequence Pre-training for Language Generation
- Achieving Human Parity on Automatic Chinese to English News Translation
- Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
- Unsupervised Machine Translation Using Monolingual Corpora Only
- A Teacher-Student Framework for Zero-Resource Neural Machine Translation
- Unsupervised Neural Machine Translation
- Universal Neural Machine Translation for Extremely Low Resource Languages
- Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input