Dependency Grammar Induction with a Neural Variational Transition-based Parser

Abstract

Dependency grammar induction is the task of learning dependency syntax without annotated training data. Traditional graph-based models with global inference achieve state-of-the-art results on this task but they require $O(n^3)$ run time. Transition-based models enable faster inference with $O(n)$ time complexity, but their performance still lags behind. In this work, we propose a neural transition-based parser for dependency grammar induction, whose inference procedure utilizes rich neural features with $O(n)$ time complexity. We train the parser with an integration of variational inference, posterior regularization and variance reduction techniques. The resulting framework outperforms previous unsupervised transition-based dependency parsers and achieves performance comparable to graph-based models, both on the English Penn Treebank and on the Universal Dependency Treebank. In an empirical comparison, we show that our approach substantially increases parsing speed over graph-based models.

arXiv commentAAAI-19

References in corpus (9)

• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Enriching Word Vectors with Subword Information
• Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression
• Unsupervised Dependency Parsing: Let's Use Supervised Parsers
• Fast(er) Exact Decoding and Global Training for Transition-Based Dependency Parsing via a Minimal Feature Set
• A Generative Parser with a Discriminative Recognition Algorithm
• CRF Autoencoder for Unsupervised Dependency Parsing
• Global Transition-based Non-projective Dependency Parsing

Cited by in corpus (1)

• Guiding Symbolic Natural Language Grammar Induction via Transformer-Based Sequence Probabilities