A Retrieve-and-Edit Framework for Predicting Structured Outputs

Abstract

For the task of generating complex outputs such as source code, editing existing outputs can be easier than generating complex outputs from scratch. With this motivation, we propose an approach that first retrieves a training example based on the input (e.g., natural language description) and then edits it to the desired output (e.g., code). Our contribution is a computationally efficient method for learning a retrieval model that embeds the input in a task-dependent way without relying on a hand-crafted metric or incurring the expense of jointly training the retriever with the editor. Our retrieve-and-edit framework can be applied on top of any base model. We show that on a new autocomplete task for GitHub Python code and the Hearthstone cards benchmark, retrieve-and-edit significantly boosts the performance of a vanilla sequence-to-sequence model on both tasks.

arXiv commentTo appear, NeurIPS 2018

Cited by in corpus (40)

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• REALM: Retrieval-Augmented Language Model Pre-Training
• Deep Learning-based Software Engineering: Progress, Challenges, and Opportunities
• Learning to Represent Edits
• Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions
• Unifying Human and Statistical Evaluation for Natural Language Generation
• Controllable Semantic Parsing via Retrieval Augmentation
• Relevance Transformer: Generating Concise Code Snippets with Relevance Feedback
• Nearest Neighbor Machine Translation
• Neural Machine Translation with Monolingual Translation Memory
• Learning Sparse Prototypes for Text Generation
• Fact-based Text Editing
• Story Realization: Expanding Plot Events into Sentences
• SPoC: Search-based Pseudocode to Code
• Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
• Learning to Recombine and Resample Data for Compositional Generalization
• Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing
• Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning
• Hierarchical Context Tagging for Utterance Rewriting
• Data-to-Text Generation with Style Imitation
• An Enhanced Knowledge Injection Model for Commonsense Generation
• Generating Code with the Help of Retrieved Template Functions and Stack Overflow Answers
• Label-Agnostic Sequence Labeling by Copying Nearest Neighbors
• Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation
• Pun Generation with Surprise
• Diversifying Task-oriented Dialogue Response Generation with Prototype Guided Paraphrasing
• Constrained Language Models Yield Few-Shot Semantic Parsers
• Exploiting Cross-Modal Prediction and Relation Consistency for Semi-Supervised Image Captioning
• Controlling Text Edition by Changing Answers of Specific Questions
• Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation
• Adversarial Training for Code Retrieval with Question-Description Relevance Regularization
• Learning to Selectively Learn for Weakly-supervised Paraphrase Generation
• Capturing Structural Locality in Non-parametric Language Models
• Sequence-to-Sequence Learning with Latent Neural Grammars
• Table-to-Text Natural Language Generation with Unseen Schemas
• SHARE: a System for Hierarchical Assistive Recipe Editing
• Case-based Reasoning for Natural Language Queries over Knowledge Bases
• Data-to-text Generation by Splicing Together Nearest Neighbors
• Look-up and Adapt: A One-shot Semantic Parser
• Predict and Use Latent Patterns for Short-Text Conversation