Inducing Grammars with and for Neural Machine Translation
arXiv:1805.10850
Abstract
Machine translation systems require semantic knowledge and grammatical understanding. Neural machine translation (NMT) systems often assume this information is captured by an attention mechanism and a decoder that ensures fluency. Recent work has shown that incorporating explicit syntax alleviates the burden of modeling both types of knowledge. However, requiring parses is expensive and does not explore the question of what syntax a model needs during translation. To address both of these issues we introduce a model that simultaneously translates while inducing dependency trees. In this way, we leverage the benefits of structure while investigating what syntax NMT must induce to maximize performance. We show that our dependency trees are 1. language pair dependent and 2. improve translation quality.
arXiv commentaccepted at NMT workshop (WNMT 2018)
References in corpus (9)
- Categorical Reparameterization with Gumbel-Softmax
- A Structured Self-attentive Sentence Embedding
- Effective Approaches to Attention-based Neural Machine Translation
- A large annotated corpus for learning natural language inference
- What do Neural Machine Translation Models Learn about Morphology?
- Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
- Learning to Compose Words into Sentences with Reinforcement Learning
- Modeling Source Syntax for Neural Machine Translation
- Learning to Parse and Translate Improves Neural Machine Translation