Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations
arXiv:1809.02286
Abstract
Most existing recursive neural network (RvNN) architectures utilize only the structure of parse trees, ignoring syntactic tags which are provided as by-products of parsing. We present a novel RvNN architecture that can provide dynamic compositionality by considering comprehensive syntactic information derived from both the structure and linguistic tags. Specifically, we introduce a structure-aware tag representation constructed by a separate tag-level tree-LSTM. With this, we can control the composition function of the existing word-level tree-LSTM by augmenting the representation as a supplementary input to the gate functions of the tree-LSTM. In extensive experiments, we show that models built upon the proposed architecture obtain superior or competitive performance on several sentence-level tasks such as sentiment analysis and natural language inference when compared against previous tree-structured models and other sophisticated neural models.
AAAI 2019
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Decoupled Weight Decay Regularization
- ADADELTA: An Adaptive Learning Rate Method
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
- A large annotated corpus for learning natural language inference
- Learning to Generate Reviews and Discovering Sentiment
- Learned in Translation: Contextualized Word Vectors
- Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
- Discriminative Neural Sentence Modeling by Tree-Based Convolution
- Enhancing Sentence Embedding with Generalized Pooling
- Learning to Compose Words into Sentences with Reinforcement Learning
- Shortcut-Stacked Sentence Encoders for Multi-Domain Inference
- Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
- Latent Tree Learning with Differentiable Parsers: Shift-Reduce Parsing and Chart Parsing
- Towards Bidirectional Hierarchical Representations for Attention-Based Neural Machine Translation
- Dynamic Compositional Neural Networks over Tree Structure