Robust Neural Machine Translation with Doubly Adversarial Inputs

Abstract

Neural machine translation (NMT) often suffers from the vulnerability to noisy perturbations in the input. We propose an approach to improving the robustness of NMT models, which consists of two parts: (1) attack the translation model with adversarial source examples; (2) defend the translation model with adversarial target inputs to improve its robustness against the adversarial source inputs.For the generation of adversarial inputs, we propose a gradient-based method to craft adversarial examples informed by the translation loss over the clean inputs.Experimental results on Chinese-English and English-German translation tasks demonstrate that our approach achieves significant improvements ($2.8$ and $1.6$ BLEU points) over Transformer on standard clean benchmarks as well as exhibiting higher robustness on noisy data.

Accepted by ACL 2019

References in corpus (7)

• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Dual Learning for Machine Translation
• Sockeye: A Toolkit for Neural Machine Translation
• On Adversarial Examples for Character-Level Neural Machine Translation
• Training Deeper Neural Machine Translation Models with Transparent Attention
• Robust Neural Machine Translation with Joint Textual and Phonetic Embedding

Cited by in corpus (14)

• Natural Language Processing Advancements By Deep Learning: A Survey
• An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
• Self-supervised Pre-training with Hard Examples Improves Visual Representations
• Adversarial Attacks and Defense on Texts: A Survey
• A Reinforced Generation of Adversarial Examples for Neural Machine Translation
• Adversarial Training with Contrastive Learning in NLP
• Word Shape Matters: Robust Machine Translation with Visual Embedding
• Evaluating Saliency Methods for Neural Language Models
• How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
• From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video
• Generating Semantically Valid Adversarial Questions for TableQA
• OCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Prediction
• Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification
• Doubly-Trained Adversarial Data Augmentation for Neural Machine Translation