Learning to Embed Sentences Using Attentive Recursive Trees
arXiv:1811.02338
Abstract
Sentence embedding is an effective feature representation for most deep learning-based NLP tasks. One prevailing line of methods is using recursive latent tree-structured networks to embed sentences with task-specific structures. However, existing models have no explicit mechanism to emphasize task-informative words in the tree structure. To this end, we propose an Attentive Recursive Tree model (AR-Tree), where the words are dynamically located according to their importance in the task. Specifically, we construct the latent tree for a sentence in a proposed important-first strategy, and place more attentive words nearer to the root; thus, AR-Tree can inherently emphasize important words during the bottom-up composition of the sentence embedding. We propose an end-to-end reinforced training strategy for AR-Tree, which is demonstrated to consistently outperform, or be at least comparable to, the state-of-the-art sentence embedding methods on three sentence understanding tasks.
arXiv commentAAAI Conference of Artificial Intelligence, 2019
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- ADADELTA: An Adaptive Learning Rate Method
- Categorical Reparameterization with Gumbel-Softmax
- Convolutional Neural Network Architectures for Matching Natural Language Sentences
- A Convolutional Neural Network for Modelling Sentences
- A large annotated corpus for learning natural language inference
- Learning to Generate Reviews and Discovering Sentiment
- Attentive Pooling Networks
- Learned in Translation: Contextualized Word Vectors
- Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
- Learning to Compose Words into Sentences with Reinforcement Learning
- Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference
- Do latent tree learning models identify meaningful structure in sentences?