Zero-Resource Neural Machine Translation with Multi-Agent Communication Game

Abstract

While end-to-end neural machine translation (NMT) has achieved notable success in the past years in translating a handful of resource-rich language pairs, it still suffers from the data scarcity problem for low-resource language pairs and domains. To tackle this problem, we propose an interactive multimodal framework for zero-resource neural machine translation. Instead of being passively exposed to large amounts of parallel corpora, our learners (implemented as encoder-decoder architecture) engage in cooperative image description games, and thus develop their own image captioning or neural machine translation model from the need to communicate in order to succeed at the game. Experimental results on the IAPR-TC12 and Multi30K datasets show that the proposed learning mechanism significantly improves over the state-of-the-art methods.

arXiv commentPublished at AAAI-18

References in corpus (9)

• Sequence to Sequence Learning with Neural Networks
• Deep Residual Learning for Image Recognition
• Convolutional Sequence to Sequence Learning
• Sequence Level Training with Recurrent Neural Networks
• Dual Learning for Machine Translation
• A Teacher-Student Framework for Zero-Resource Neural Machine Translation
• Multimodal Pivots for Image Caption Translation
• Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
• Towards Multi-Agent Communication-Based Language Learning

Cited by in corpus (4)

• Meta-Learning for Low-Resource Neural Machine Translation
• Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations
• Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning
• Exploration on Grounded Word Embedding: Matching Words and Images with Image-Enhanced Skip-Gram Model