Top-down Tree Long Short-Term Memory Networks

Abstract

Long Short-Term Memory (LSTM) networks, a type of recurrent neural network with a more complex computational unit, have been successfully applied to a variety of sequence modeling tasks. In this paper we develop Tree Long Short-Term Memory (TreeLSTM), a neural network model based on LSTM, which is designed to predict a tree rather than a linear sequence. TreeLSTM defines the probability of a sentence by estimating the generation probability of its dependency tree. At each time step, a node is generated based on the representation of the generated sub-tree. We further enhance the modeling power of TreeLSTM by explicitly representing the correlations between left and right dependents. Application of our model to the MSR sentence completion challenge achieves results beyond the current state of the art. We also report results on dependency parsing reranking achieving competitive performance.

to appear in NAACL 2016; code available at https://github.com/XingxingZhang/td-treelstm

References in corpus (4)

• Sequence to Sequence Learning with Neural Networks
• Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Learning Longer Memory in Recurrent Neural Networks

Cited by in corpus (8)

• Syntax-Directed Variational Autoencoder for Structured Data
• Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
• Neural Language Modeling by Jointly Learning Syntax and Lexicon
• Direct Estimation of Regional Wall Thicknesses via Residual Recurrent Neural Network
• A Simple LSTM model for Transition-based Dependency Parsing
• A multi-level convolutional LSTM model for the segmentation of left ventricle myocardium in infarcted porcine cine MR images
• Recursive Top-Down Production for Sentence Generation with Latent Trees
• On Architectures for Including Visual Information in Neural Language Models for Image Description