Compound Probabilistic Context-Free Grammars for Grammar Induction
arXiv:1906.10225
Abstract
We study a formalization of the grammar induction problem that models sentences as being generated by a compound probabilistic context-free grammar. In contrast to traditional formulations which learn a single stochastic grammar, our grammar's rule probabilities are modulated by a per-sentence continuous latent variable, which induces marginal dependencies beyond the traditional context-free assumptions. Inference in this grammar is performed by collapsed variational inference, in which an amortized variational posterior is placed on the continuous variable, and the latent trees are marginalized out with dynamic programming. Experiments on English and Chinese show the effectiveness of our approach compared to recent state-of-the-art methods when evaluated on unsupervised parsing.
ACL 2019
References in corpus (10)
- Auto-Encoding Variational Bayes
- Variational Inference with Normalizing Flows
- Character-Aware Neural Language Models
- Exploring the Limits of Language Modeling
- Layer Normalization
- Gradient Descent Provably Optimizes Over-parameterized Neural Networks
- Improving Variational Inference with Inverse Autoregressive Flow
- Visually Grounded Neural Syntax Acquisition
- Long Short-Term Memory Over Tree Structures
- Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders
Cited by in corpus (5)
- Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads
- Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach
- Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales
- Semi-supervised Autoencoding Projective Dependency Parsing
- Learning Algebraic Recombination for Compositional Generalization