Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder

Abstract

Human annotation for syntactic parsing is expensive, and large resources are available only for a fraction of languages. A question we ask is whether one can leverage abundant unlabeled texts to improve syntactic parsers, beyond just using the texts to obtain more generalisable lexical features (i.e. beyond word embeddings). To this end, we propose a novel latent-variable generative model for semi-supervised syntactic dependency parsing. As exact inference is intractable, we introduce a differentiable relaxation to obtain approximate samples and compute gradients with respect to the parser parameters. Our method (Differentiable Perturb-and-Parse) relies on differentiable dynamic programming over stochastically perturbed edge scores. We demonstrate effectiveness of our approach with experiments on English, French and Swedish.

arXiv commentAccepted at ICLR 2019

Cited by in corpus (15)

• Zero-Resource Knowledge-Grounded Dialogue Generation
• Gradient Estimation with Stochastic Softmax Tricks
• Speaker-Oriented Latent Structures for Dialogue-Based Relation Extraction
• Direct Optimization through $\arg \max$ for Discrete Variational Auto-Encoder
• Unsupervised Recurrent Neural Network Grammars
• Learning with Algorithmic Supervision via Continuous Relaxations
• Cooperative Learning of Disjoint Syntax and Semantics
• Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach
• Sparse Communication via Mixed Distributions
• Reconciling the Discrete-Continuous Divide: Towards a Mathematical Theory of Sparse Communication
• A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsing
• Challenging the Semi-Supervised VAE Framework for Text Classification
• Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spaces
• Understanding in Artificial Intelligence
• Controlling the Interaction Between Generation and Inference in Semi-Supervised Variational Autoencoders Using Importance Weighting