Long Short-Term Memory Over Tree Structures

Abstract

The chain-structured long short-term memory (LSTM) has showed to be effective in a wide range of problems such as speech recognition and machine translation. In this paper, we propose to extend it to tree structures, in which a memory cell can reflect the history memories of multiple child cells or multiple descendant cells in a recursive process. We call the model S-LSTM, which provides a principled way of considering long-distance interaction over hierarchies, e.g., language or image parse structures. We leverage the models for semantic composition to understand the meaning of text, a fundamental problem in natural language understanding, and show that it outperforms a state-of-the-art recursive model by replacing its composition layers with the S-LSTM memory blocks. We also show that utilizing the given structures is helpful in achieving a performance better than that without considering the structures.

On February 6th, 2015, this work was submitted to the International Conference on Machine Learning (ICML)

References in corpus (1)

• Show and Tell: A Neural Image Caption Generator

Cited by in corpus (17)

• Contextual LSTM (CLSTM) models for Large scale NLP tasks
• Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path
• Aspect Level Sentiment Classification with Deep Memory Network
• Dependency-based Convolutional Neural Networks for Sentence Embedding
• Discriminative Neural Sentence Modeling by Tree-Based Convolution
• A Comprehensive Survey on Hardware-Aware Neural Architecture Search
• Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Data
• Gated Recurrent Neural Tensor Network
• Bidirectional Tree-Structured LSTM with Head Lexicalization
• Compositional Distributional Semantics with Long Short Term Memory
• DAG-Recurrent Neural Networks For Scene Labeling
• Distilling Word Embeddings: An Encoding Approach
• Tree Memory Networks for Modelling Long-term Temporal Dependencies
• Compound Probabilistic Context-Free Grammars for Grammar Induction
• Unsupervised Recurrent Neural Network Grammars
• Tree-Structured Neural Machine for Linguistics-Aware Sentence Generation
• Sequence-to-Sequence Learning with Latent Neural Grammars