Select and Attend: Towards Controllable Content Selection in Text Generation

Abstract

Many text generation tasks naturally contain two steps: content selection and surface realization. Current neural encoder-decoder models conflate both steps into a black-box architecture. As a result, the content to be described in the text cannot be explicitly controlled. This paper tackles this problem by decoupling content selection from the decoder. The decoupled content selection is human interpretable, whose value can be manually manipulated to control the content of generated text. The model can be trained end-to-end without human annotations by maximizing a lower bound of the marginal likelihood. We further propose an effective way to trade-off between performance and controllability with a single adjustable hyperparameter. In both data-to-text and headline generation tasks, our model achieves promising results, paving the way for controllable content selection in text generation.

arXiv commentEMNLP 2019

References in corpus (18)

• Sequence to Sequence Learning with Neural Networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Neural Discrete Representation Learning
• Building Natural-Language Generation Systems
• SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
• The influence of feature selection methods on accuracy, stability and interpretability of molecular signatures
• Selective Encoding for Abstractive Sentence Summarization
• Neural Variational Inference and Learning in Belief Networks
• Fast Decoding in Sequence Models using Discrete Latent Variables
• The Optimal Reward Baseline for Gradient-Based Reinforcement Learning
• REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
• Variational inference for Monte Carlo objectives
• Latent Alignment and Variational Attention
• Techniques for Learning Binary Stochastic Feedforward Neural Networks
• The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models
• Focused Hierarchical RNNs for Conditional Sequence Processing
• Data-to-Text Generation with Style Imitation
• Show, Control and Tell: A Framework for Generating Controllable and Grounded Captions

Cited by in corpus (4)

• Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence
• Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View
• Diversifying Dialogue Generation with Non-Conversational Text
• Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task