Dependency Parsing with LSTMs: An Empirical Evaluation
arXiv:1604.06529
Abstract
We propose a transition-based dependency parser using Recurrent Neural Networks with Long Short-Term Memory (LSTM) units. This extends the feedforward neural network parser of Chen and Manning (2014) and enables modelling of entire sequences of shift/reduce transition decisions. On the Google Web Treebank, our LSTM parser is competitive with the best feedforward parser on overall accuracy and notably achieves more than 3% improvement for long-range dependencies, which has proved difficult for previous transition-based parsers due to error propagation and limited context information. Our findings additionally suggest that dropout regularisation on the embedding layer is crucial to improve the LSTM's generalisation.
arXiv comment7 pages, 4 figures
References in corpus (8)
- Sequence to Sequence Learning with Neural Networks
- LSTM: A Search Space Odyssey
- ADADELTA: An Adaptive Learning Rate Method
- Recurrent Neural Network Regularization
- Transition-Based Dependency Parsing with Stack Long Short-Term Memory
- Grammar as a Foreign Language
- Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations
- Structured Training for Neural Network Transition-Based Parsing