The University of Edinburgh's Neural MT Systems for WMT17
arXiv:1708.00726
Abstract
This paper describes the University of Edinburgh's submissions to the WMT17 shared news translation and biomedical translation tasks. We participated in 12 translation directions for news, translating between English and Czech, German, Latvian, Russian, Turkish and Chinese. For the biomedical task we submitted systems for English to Czech, German, Polish and Romanian. Our systems are neural machine translation systems trained with Nematus, an attentional encoder-decoder. We follow our setup from last year and build BPE-based models with parallel and back-translated monolingual training data. Novelties this year include the use of deep architectures, layer normalization, and more compact models due to weight tying and improvements in BPE segmentations. We perform extensive ablative experiments, reporting on the effectivenes of layer normalization, deep architectures, and different ensembling techniques.
arXiv commentWMT 2017 shared task track; for Bibtex, see http://homepages.inf.ed.ac.uk/rsennric/bib.html#uedin-nmt:2017
Cited by in corpus (12)
- The Natural Language Decathlon: Multitask Learning as Question Answering
- Emergent Translation in Multi-Agent Communication
- Checkpoint Ensembles: Ensemble Methods from a Single Training Process
- A Study of Reinforcement Learning for Neural Machine Translation
- Hybrid Self-Attention Network for Machine Translation
- Learning to Discriminate Noises for Incorporating External Information in Neural Machine Translation
- Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation
- Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation
- Fine-grained Human Evaluation of Transformer and Recurrent Approaches to Neural Machine Translation for English-to-Chinese
- Synchronous Bidirectional Neural Machine Translation
- Modeling Past and Future for Neural Machine Translation
- Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation