Structured Training for Neural Network Transition-Based Parsing

Abstract

We present structured perceptron training for neural network transition-based dependency parsing. We learn the neural network representation using a gold corpus augmented by a large number of automatically parsed sentences. Given this fixed network representation, we learn a final layer using the structured perceptron with beam-search decoding. On the Penn Treebank, our parser reaches 94.26% unlabeled and 92.41% labeled attachment accuracy, which to our knowledge is the best accuracy on Stanford Dependencies to date. We also provide in-depth ablative analysis to determine which aspects of our model provide the largest gains in accuracy.

References in corpus (3)

• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Grammar as a Foreign Language
• A Linear Observed Time Statistical Parser Based on Maximum Entropy Models

Cited by in corpus (25)

• Recent Trends in Deep Learning Based Natural Language Processing
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs
• A Survey of Deep Learning for Scientific Discovery
• Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations
• A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
• Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement
• Exploiting Multi-typed Treebanks for Parsing with Deep Multi-task Learning
• Character Composition Model with Convolutional Neural Networks for Dependency Parsing on Morphologically Rich Languages
• Bi-directional Attention with Agreement for Dependency Parsing
• A* CCG Parsing with a Supertag and Dependency Factored Model
• A Bayesian Model for Generative Transition-based Dependency Parsing
• Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles
• A Simple LSTM model for Transition-based Dependency Parsing
• Incremental Parsing with Minimal Features Using Bi-Directional LSTM
• An empirical study for Vietnamese dependency parsing
• Dependency Parsing as Head Selection
• A Fast, Compact, Accurate Model for Language Identification of Codemixed Text
• Sentence Encoding with Tree-constrained Relation Networks
• Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
• Embedding Words and Senses Together via Joint Knowledge-Enhanced Training
• Dependency Parsing with LSTMs: An Empirical Evaluation
• Stack-propagation: Improved Representation Learning for Syntax
• Tackling Error Propagation through Reinforcement Learning: A Case of Greedy Dependency Parsing
• Semi-supervised Autoencoding Projective Dependency Parsing