Visual Cues and Error Correction for Translation Robustness

Abstract

Neural Machine Translation models are sensitive to noise in the input texts, such as misspelled words and ungrammatical constructions. Existing robustness techniques generally fail when faced with unseen types of noise and their performance degrades on clean texts. In this paper, we focus on three types of realistic noise that are commonly generated by humans and introduce the idea of visual context to improve translation robustness for noisy texts. In addition, we describe a novel error correction training regime that can be used as an auxiliary task to further improve translation robustness. Experiments on English-French and English-German translation show that both multimodal and error correction components improve model robustness to noisy texts, while still retaining translation quality on clean texts.

arXiv commentAccepted at Findings of EMNLP 2021; add acknowledgements

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Synthetic and Natural Noise Both Break Neural Machine Translation
• On Adversarial Examples for Character-Level Neural Machine Translation
• Multimodal Attention for Neural Machine Translation
• Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back Translation
• Predicting Actions to Help Predict Translations
• Towards Multimodal Simultaneous Neural Machine Translation
• Addressing the Vulnerability of NMT in Input Perturbations
• Transformer-based Cascaded Multimodal Speech Translation