Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder

Abstract

Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs. Existing works usually ignore the context that is not explicitly provided, resulting in a context-independent semantic representation that struggles to support the generation. To address this, we propose an approach that automatically finds evidence for an event from a large text corpus, and leverages the evidence to guide the generation of inferential texts. Our approach works in an encoder-decoder manner and is equipped with a Vector Quantised-Variational Autoencoder, where the encoder outputs representations from a distribution over discrete variables. Such discrete representations enable automatically selecting relevant evidence, which not only facilitates evidence-aware generation, but also provides a natural way to uncover rationales behind the generation. Our approach provides state-of-the-art performance on both Event2Mind and ATOMIC datasets. More importantly, we find that with discrete representations, our model selectively uses evidence to generate different inferential texts.

Accepted by ACL 2020

References in corpus (12)

• Neural Discrete Representation Learning
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
• Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
• A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression
• Variational Neural Machine Translation
• Constructing Narrative Event Evolutionary Graph for Script Event Prediction
• Variational Recurrent Neural Machine Translation
• Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning
• Modeling Event Background for If-Then Commonsense Reasoning Using Context-aware Variational Autoencoder