Controllable Neural Story Plot Generation via Reward Shaping

Abstract

Language-modeling--based approaches to story plot generation attempt to construct a plot by sampling from a language model (LM) to predict the next character, word, or sentence to add to the story. LM techniques lack the ability to receive guidance from the user to achieve a specific goal, resulting in stories that don't have a clear sense of progression and lack coherence. We present a reward-shaping technique that analyzes a story corpus and produces intermediate rewards that are backpropagated into a pre-trained LM in order to guide the model towards a given goal. Automated evaluations show our technique can create a model that generates story plots which consistently achieve a specified goal. Human-subject studies show that the generated stories have more plausible event ordering than baseline plot generation techniques.

Pradyumna Tambwekar & Murtaza Dhuliawala contributed equally

References in corpus (3)

• Sequence to Sequence Learning with Neural Networks
• Convolutional Sequence to Sequence Learning
• Event Representations for Automated Story Generation with Deep Neural Nets

Cited by in corpus (18)

• The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
• Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
• Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges
• PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking
• CharacterChat: Supporting the Creation of Fictional Characters through Conversation and Progressive Manifestation with a Chatbot
• Creativity and Machine Learning: A Survey
• A Distributional Approach to Controlled Text Generation
• Controllable Data Generation by Deep Learning: A Review
• Story Realization: Expanding Plot Events into Sentences
• TopNet: Learning from Neural Topic Model to Generate Long Stories
• What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
• Automatic Story Generation: Challenges and Attempts
• Cue Me In: Content-Inducing Approaches to Interactive Story Generation
• Energy-Based Models for Code Generation under Compilability Constraints
• Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs
• GraphPlan: Story Generation by Planning with Event Graph
• Changing the Mind of Transformers for Topically-Controllable Language Generation
• CoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding