Cooperative Learning of Disjoint Syntax and Semantics
arXiv:1902.09393
Abstract
There has been considerable attention devoted to models that learn to jointly infer an expression's syntactic structure and its semantics. Yet, \citet{NangiaB18} has recently shown that the current best systems fail to learn the correct parsing strategy on mathematical expressions generated from a simple context-free grammar. In this work, we present a recursive model inspired by \newcite{ChoiYL18} that reaches near perfect accuracy on this task. Our model is composed of two separated modules for syntax and semantics. They are cooperatively trained with standard continuous and discrete optimization schemes. Our model does not require any linguistic structure for supervision and its recursive nature allows for out-of-domain generalization with little loss in performance. Additionally, our approach performs competitively on several natural language tasks, such as Natural Language Inference or Sentiment Analysis.
arXiv commentThe paper was accepted at NAACL-HLT 2019
References in corpus (13)
- ADADELTA: An Adaptive Learning Rate Method
- Categorical Reparameterization with Gumbel-Softmax
- Enhanced LSTM for Natural Language Inference
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Learning to Generate Reviews and Discovering Sentiment
- Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
- REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
- Deep Learning with Dynamic Computation Graphs
- Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
- Automatically Composing Representation Transformations as a Means for Generalization
- Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder
- Learning to Compose Skills
- Latent Tree Learning with Differentiable Parsers: Shift-Reduce Parsing and Chart Parsing