Improving the Robustness of Speech Translation

Abstract

Although neural machine translation (NMT) has achieved impressive progress recently, it is usually trained on the clean parallel data set and hence cannot work well when the input sentence is the production of the automatic speech recognition (ASR) system due to the enormous errors in the source. To solve this problem, we propose a simple but effective method to improve the robustness of NMT in the case of speech translation. We simulate the noise existing in the realistic output of the ASR system and inject them into the clean parallel data so that NMT can work under similar word distributions during training and testing. Besides, we also incorporate the Chinese Pinyin feature which is easy to get in speech translation to further improve the translation performance. Experiment results show that our method has a more stable performance and outperforms the baseline by an average of 3.12 BLEU on multiple noisy test sets, even while achieves a generalization improvement on the WMT'17 Chinese-English test set.

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• Improving neural networks by preventing co-adaptation of feature detectors
• Synthetic and Natural Noise Both Break Neural Machine Translation
• Towards Robust Neural Machine Translation
• Neural Lattice-to-Sequence Models for Uncertain Inputs
• Disentangling ASR and MT Errors in Speech Translation

Cited by in corpus (5)

• DuTongChuan: Context-aware Translation Model for Simultaneous Interpreting
• Pre-training for Spoken Language Understanding with Joint Textual and Phonetic Representation Learning
• Robust Neural Machine Translation with Joint Textual and Phonetic Embedding
• An Approach to Improve Robustness of NLP Systems against ASR Errors
• Modeling Homophone Noise for Robust Neural Machine Translation