Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Stories

Abstract

Measuring event salience is essential in the understanding of stories. This paper takes a recent unsupervised method for salience detection derived from Barthes Cardinal Functions and theories of surprise and applies it to longer narrative forms. We improve the standard transformer language model by incorporating an external knowledgebase (derived from Retrieval Augmented Generation) and adding a memory mechanism to enhance performance on longer works. We use a novel approach to derive salience annotation using chapter-aligned summaries from the Shmoop corpus for classic literary works. Our evaluation against this data demonstrates that our salience detection model improves performance over and above a non-knowledgebase and memory augmented language model, both of which are crucial to this improvement.

arXiv commentAccepted to the EMNLP 2021 Conference as a long-paper, 9 pages, 15 pages with appendices and references, 2 figures, 4 tables

References in corpus (6)

• Leveraging BERT for Extractive Text Summarization on Lectures
• A Practical Survey on Faster and Lighter Transformers
• Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering
• Retrieval Augmentation Reduces Hallucination in Conversation
• The Shmoop Corpus: A Dataset of Stories with Loosely Aligned Summaries
• Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning