Plot and Rework: Modeling Storylines for Visual Storytelling
arXiv:2105.06950
Abstract
Writing a coherent and engaging story is not easy. Creative writers use their knowledge and worldview to put disjointed elements together to form a coherent storyline, and work and rework iteratively toward perfection. Automated visual storytelling (VIST) models, however, make poor use of external knowledge and iterative generation when attempting to create stories. This paper introduces PR-VIST, a framework that represents the input image sequence as a story graph in which it finds the best path to form a storyline. PR-VIST then takes this path and learns to generate the final story via an iterative training process. This framework produces stories that are superior in terms of diversity, coherence, and humanness, per both automatic and human evaluations. An ablation study shows that both plotting and reworking contribute to the model's superiority.
arXiv comment9 pages, ACL-IJCNLP 2021 Findings
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold
- GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation
- UHop: An Unrestricted-Hop Relation Extraction Framework for Knowledge-Based Question Answering
- Contextualize, Show and Tell: A Neural Visual Storyteller
- Using Inter-Sentence Diverse Beam Search to Reduce Redundancy in Visual Storytelling
- What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
- Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication