On learning an interpreted language with recurrent models
arXiv:1809.04128
Abstract
Can recurrent neural nets, inspired by human sequential data processing, learn to understand language? We construct simplified datasets reflecting core properties of natural language as modeled in formal syntax and semantics: recursive syntactic structure and compositionality. We find LSTM and GRU networks to generalise to compositional interpretation well, but only in the most favorable learning settings, with a well-paced curriculum, extensive training data, and left-to-right (but not right-to-left) composition.
arXiv commentTo appear in Computational Linguistics
References in corpus (7)
- Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
- Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
- Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans
- Memorize or generalize? Searching for a compositional RNN in a haystack
- Discovering the Compositional Structure of Vector Representations with Role Learning Networks
- Learning compositionally through attentive guidance
- What Meaning-Form Correlation Has to Compose With