Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

Abstract

We propose a hierarchically structured reinforcement learning approach to address the challenges of planning for generating coherent multi-sentence stories for the visual storytelling task. Within our framework, the task of generating a story given a sequence of images is divided across a two-level hierarchical decoder. The high-level decoder constructs a plan by generating a semantic concept (i.e., topic) for each image in sequence. The low-level decoder generates a sentence for each image using a semantic compositional network, which effectively grounds the sentence generation conditioned on the topic. The two decoders are jointly trained end-to-end using reinforcement learning. We evaluate our model on the visual storytelling (VIST) dataset. Empirical results from both automatic and human evaluations demonstrate that the proposed hierarchically structured reinforced training achieves significantly better performance compared to a strong flat deep reinforcement learning baseline.

arXiv commentAccepted to AAAI 2019

References in corpus (13)

• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Microsoft COCO Captions: Data Collection and Evaluation Server
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• A Deep Reinforced Model for Abstractive Summarization
• Improved Image Captioning via Policy Gradient optimization of SPIDEr
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Sequence to Sequence -- Video to Text
• Describing Videos by Exploiting Temporal Structure
• Language Models for Image Captioning: The Quirks and What Works
• Visual Storytelling
• Deep Reinforcement Learning-based Image Captioning with Embedding Reward

Cited by in corpus (5)

• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
• Improving Sequence-to-Sequence Learning via Optimal Transport
• Informative Visual Storytelling with Cross-modal Rules
• StoryGAN: A Sequential Conditional GAN for Story Visualization
• Visual Storytelling via Predicting Anchor Word Embeddings in the Stories