What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
arXiv:1909.05316
Abstract
Previous storytelling approaches mostly focused on optimizing traditional metrics such as BLEU, ROUGE and CIDEr. In this paper, we re-examine this problem from a different angle, by looking deep into what defines a realistically-natural and topically-coherent story. To this end, we propose three assessment criteria: relevance, coherence and expressiveness, which we observe through empirical analysis could constitute a "high-quality" story to the human eye. Following this quality guideline, we propose a reinforcement learning framework, ReCo-RL, with reward functions designed to capture the essence of these quality criteria. Experiments on the Visual Storytelling Dataset (VIST) with both automatic and human evaluations demonstrate that our ReCo-RL model achieves better performance than state-of-the-art baselines on both traditional metrics and the proposed new criteria.
arXiv commentAccepted paper in Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI) 2020
References in corpus (13)
- Sequence to Sequence Learning with Neural Networks
- Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- A Deep Reinforced Model for Abstractive Summarization
- Professor Forcing: A New Algorithm for Training Recurrent Networks
- Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
- A Hierarchical Neural Autoencoder for Paragraphs and Documents
- UNITER: UNiversal Image-TExt Representation Learning
- Hierarchical Neural Story Generation
- Adversarial Feature Matching for Text Generation
- Visual Storytelling
- Controllable Neural Story Plot Generation via Reward Shaping
- Toward Diverse Text Generation with Inverse Reinforcement Learning
- Domain Adaptive Text Style Transfer