Data-to-text Generation by Splicing Together Nearest Neighbors

Abstract

We propose to tackle data-to-text generation tasks by directly splicing together retrieved segments of text from "neighbor" source-target pairs. Unlike recent work that conditions on retrieved neighbors but generates text token-by-token, left-to-right, we learn a policy that directly manipulates segments of neighbor text, by inserting or replacing them in partially constructed generations. Standard techniques for training such a policy require an oracle derivation for each generation, and we prove that finding the shortest such derivation can be reduced to parsing under a particular weighted context-free grammar. We find that policies learned in this way perform on par with strong baselines in terms of automatic and human evaluation, but allow for more interpretable and controllable generation.

arXiv commentEMNLP 2021; figures updated/improved

References in corpus (14)

• Sequence to Sequence Learning with Neural Networks
• Decoupled Weight Decay Regularization
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Non-Autoregressive Neural Machine Translation
• MBT: A Memory-Based Part of Speech Tagger-Generator
• Insertion Transformer: Flexible Sequence Generation via Insertion Operations
• A Retrieve-and-Edit Framework for Predicting Structured Outputs
• Pre-training via Paraphrasing
• Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems
• Non-Monotonic Sequential Text Generation
• Syntactically Supervised Transformers for Faster Neural Machine Translation
• Learning Sparse Prototypes for Text Generation
• A Simple Approach to Case-Based Reasoning in Knowledge Bases
• Exemplar Auditing for Multi-Label Biomedical Text Classification