Training with Exploration Improves a Greedy Stack-LSTM Parser

Abstract

We adapt the greedy Stack-LSTM dependency parser of Dyer et al. (2015) to support a training-with-exploration procedure using dynamic oracles(Goldberg and Nivre, 2013) instead of cross-entropy minimization. This form of training, which accounts for model predictions at training time rather than assuming an error-free action history, improves parsing accuracies for both English and Chinese, obtaining very strong results for both languages. We discuss some modifications needed in order to get training with exploration to work well for a probabilistic neural-network.

In proceedings of EMNLP 2016

References in corpus (4)

• Sequence to Sequence Learning with Neural Networks
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Search-based Structured Prediction
• Minimum Risk Training for Neural Machine Translation

Cited by in corpus (15)

• Glyce: Glyph-vectors for Chinese Character Representations
• Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations
• Neural Probabilistic Model for Non-projective MST Parsing
• On-the-fly Operation Batching in Dynamic Computation Graphs
• SEARNN: Training RNNs with Global-Local Losses
• A Minimal Span-Based Neural Constituency Parser
• What Do Recurrent Neural Network Grammars Learn About Syntax?
• Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles
• Parsing Tweets into Universal Dependencies
• Distilling Knowledge for Search-based Structured Prediction
• Neural Architectures for Named Entity Recognition
• Non-Projective Dependency Parsing via Latent Heads Representation (LHR)
• In-Order Transition-based Constituent Parsing
• Hybrid Oracle: Making Use of Ambiguity in Transition-based Chinese Dependency Parsing
• Imitation Learning for Neural Morphological String Transduction