Improving Back-Translation with Uncertainty-based Confidence Estimation
arXiv:1909.00157
Abstract
While back-translation is simple and effective in exploiting abundant monolingual corpora to improve low-resource neural machine translation (NMT), the synthetic bilingual corpora generated by NMT models trained on limited authentic bilingual data are inevitably noisy. In this work, we propose to quantify the confidence of NMT model predictions based on model uncertainty. With word- and sentence-level confidence measures based on uncertainty, it is possible for back-translation to better cope with noise in synthetic bilingual corpora. Experiments on Chinese-English and English-German translation tasks show that uncertainty-based confidence estimation significantly improves the performance of back-translation.
arXiv commentEMNLP 2019
References in corpus (12)
- Sequence to Sequence Learning with Neural Networks
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- Weight Uncertainty in Neural Networks
- Achieving Human Parity on Automatic Chinese to English News Translation
- Regularizing Neural Networks by Penalizing Confident Output Distributions
- Deep and Confident Prediction for Time Series at Uber
- Investigating Backtranslation in Neural Machine Translation
- A Teacher-Student Framework for Zero-Resource Neural Machine Translation
- THUMT: An Open Source Toolkit for Neural Machine Translation
- Modeling Uncertainty with Hedged Instance Embedding
- Confidence through Attention
Cited by in corpus (5)
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Uncertainty Estimation in Autoregressive Structured Prediction
- Neural Machine Translation: A Review of Methods, Resources, and Tools
- Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
- AR: Auto-Repair the Synthetic Data for Neural Machine Translation