Semi-supervised Autoencoding Projective Dependency Parsing

Abstract

We describe two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing. The first model is a Locally Autoencoding Parser (LAP) encoding the input using continuous latent variables in a sequential manner; The second model is a Globally Autoencoding Parser (GAP) encoding the input into dependency trees as latent variables, with exact inference. Both models consist of two parts: an encoder enhanced by deep neural networks (DNN) that can utilize the contextual information to encode the input into latent variables, and a decoder which is a generative model able to reconstruct the input. Both LAP and GAP admit a unified structure with different loss functions for labeled and unlabeled data with shared parameters. We conducted experiments on WSJ and UD dependency parsing data sets, showing that our models can exploit the unlabeled data to improve the performance given a limited amount of labeled data, and outperform a previously proposed semi-supervised model.

arXiv commentCOLING 2020

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Sequence-to-Sequence Learning as Beam-Search Optimization
• Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
• Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations
• Structured Training for Neural Network Transition-Based Parsing
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression
• Stack-Pointer Networks for Dependency Parsing
• Compound Probabilistic Context-Free Grammars for Grammar Induction