Constrained Language Models Yield Few-Shot Semantic Parsers
arXiv:2104.08768
Abstract
We explore the use of large pretrained language models as few-shot semantic parsers. The goal in semantic parsing is to generate a structured meaning representation given a natural language input. However, language models are trained to generate natural language. To bridge the gap, we use language models to paraphrase inputs into a controlled sublanguage resembling English that can be automatically mapped to a target meaning representation. Our results demonstrate that with only a small amount of data and very little code to convert into English-like representations, our blueprint for rapidly bootstrapping semantic parsers leads to surprisingly effective performance on multiple community tasks, greatly exceeding baseline methods also trained on the same limited data.
EMNLP 2021. Code is available at https://github.com/microsoft/semantic_parsing_with_constrained_lm
References in corpus (7)
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Prefix-Tuning: Optimizing Continuous Prompts for Generation
- A Retrieve-and-Edit Framework for Predicting Structured Outputs
- GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing
- Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations
- Unnatural Language Processing: Bridging the Gap Between Synthetic and Natural Language Data
- Low-Resource Task-Oriented Semantic Parsing via Intrinsic Modeling