Bag-of-Words as Target for Neural Machine Translation

Abstract

A sentence can be translated into more than one correct sentences. However, most of the existing neural machine translation models only use one of the correct translations as the targets, and the other correct sentences are punished as the incorrect sentences in the training stage. Since most of the correct translations for one sentence share the similar bag-of-words, it is possible to distinguish the correct translations from the incorrect ones by the bag-of-words. In this paper, we propose an approach that uses both the sentences and the bag-of-words as targets in the training stage, in order to encourage the model to generate the potentially correct sentences that are not appeared in the training set. We evaluate our model on a Chinese-English translation dataset, and experiments show our model outperforms the strong baselines by the BLEU score of 4.55.

accepted by ACL 2018

References in corpus (10)

• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• On the difficulty of training Recurrent Neural Networks
• Convolutional Sequence to Sequence Learning
• meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting
• DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
• Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation
• Label Embedding Network: Learning Label Representation for Soft Training of Deep Networks
• Interactive Attention for Neural Machine Translation
• Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation

Cited by in corpus (8)

• Glyce: Glyph-vectors for Chinese Character Representations
• Is Word Segmentation Necessary for Deep Learning of Chinese Representations?
• Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation
• KECRS: Towards Knowledge-Enriched Conversational Recommendation System
• Identifying extra high frequency gravitational waves generated from oscillons with cuspy potentials using deep neural networks
• Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation
• Exchange-Based Diffusion in Hb-Graphs: Highlighting Complex Relationships
• Regularized Context Gates on Transformer for Machine Translation