Hierarchical Neural Story Generation

Abstract

We explore story generation: creative systems that can build coherent and fluent passages of text about a topic. We collect a large dataset of 300K human-written stories paired with writing prompts from an online forum. Our dataset enables hierarchical story generation, where the model first generates a premise, and then transforms it into a passage of text. We gain further improvements with a novel form of model fusion that improves the relevance of the story to the prompt, and adding a new gated multi-scale self-attention mechanism to model long-range context. Experiments show large improvements over strong baselines on both automated and human evaluations. Human judges prefer stories generated by our approach to those from a strong non-hierarchical model by a factor of two to one.

References in corpus (1)

• Towards better decoding and language model integration in sequence to sequence models

Cited by in corpus (71)

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• Towards a Human-like Open-Domain Chatbot
• Neural Text Generation with Unlikelihood Training
• Evaluation of Text Generation: A Survey
• Plug and Play Language Models: A Simple Approach to Controlled Text Generation
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Understanding Back-Translation at Scale
• Zero-shot Text Classification With Generative Language Models
• Recursively Summarizing Books with Human Feedback
• The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey
• Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds
• Unsupervised Paraphrase Generation using Pre-trained Language Models
• Plan-And-Write: Towards Better Automatic Storytelling
• The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models
• Wordcraft: a Human-AI Collaborative Editor for Story Writing
• Knowledge-Enhanced Personalized Review Generation with Capsule Graph Neural Network
• Narrative Interpolation for Generating and Understanding Stories
• Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context
• Neural Language Generation: Formulation, Methods, and Evaluation
• Encoder-Agnostic Adaptation for Conditional Language Generation
• Trading Off Diversity and Quality in Natural Language Generation
• Document Ranking with a Pretrained Sequence-to-Sequence Model
• TaleStream: Supporting Story Ideation with Trope Knowledge
• Story Realization: Expanding Plot Events into Sentences
• Intersectional Bias in Causal Language Models
• Analysis of diversity-accuracy tradeoff in image captioning
• Enabling Language Models to Fill in the Blanks
• Learning to Respond with Your Favorite Stickers: A Framework of Unifying Multi-Modality and User Preference in Multi-Turn Dialog
• Transform and Tell: Entity-Aware News Image Captioning
• Induction and Reference of Entities in a Visual Story
• BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
• ParaCNN: Visual Paragraph Generation via Adversarial Twin Contextual CNNs
• Automatic Story Generation: Challenges and Attempts
• Consistency of a Recurrent Language Model With Respect to Incomplete Decoding
• What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
• Story Ending Generation with Incremental Encoding and Commonsense Knowledge
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
• Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification
• Shopping in the Multiverse: A Counterfactual Approach to In-Session Attribution
• Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models
• Improving Language Generation with Sentence Coherence Objective
• MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding
• Back-translation for Large-Scale Multilingual Machine Translation
• Extractive Summary as Discrete Latent Variables
• Pun Generation with Surprise
• Generating similes effortlessly like a Pro: A Style Transfer Approach for Simile Generation
• Reverse Engineering Configurations of Neural Text Generation Models
• UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation
• Holdout-Based Fidelity and Privacy Assessment of Mixed-Type Synthetic Data
• OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics
• THEaiTRE: Artificial Intelligence to Write a Theatre Play
• Enhancing Self-Disclosure In Neural Dialog Models By Candidate Re-ranking
• Towards Discourse Parsing-inspired Semantic Storytelling
• Generating Interactive Worlds with Text
• "Nice Try, Kiddo": Investigating Ad Hominems in Dialogue Responses
• Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation
• Collocation2Text: Controllable Text Generation from Guide Phrases in Russian
• Long-range and hierarchical language predictions in brains and algorithms
• Semantic-Enhanced Explainable Finetuning for Open-Domain Dialogues
• Generating Dialogue Responses from a Semantic Latent Space
• Relevance-Promoting Language Model for Short-Text Conversation
• Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation
• Learning Representations and Agents for Information Retrieval
• THEaiTRE 1.0: Interactive generation of theatre play scripts
• Fact-based Dialogue Generation with Convergent and Divergent Decoding
• On How Users Edit Computer-Generated Visual Stories
• Lightweight Decoding Strategies for Increasing Specificity
• Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation
• Controllable Multi-Character Psychology-Oriented Story Generation
• Augmenting Machine Learning with Information Retrieval to Recommend Real Cloned Code Methods for Code Completion