Video Captioning via Hierarchical Reinforcement Learning

Abstract

Video captioning is the task of automatically generating a textual description of the actions in a video. Although previous work (e.g. sequence-to-sequence model) has shown promising results in abstracting a coarse description of a short video, it is still very challenging to caption a video containing multiple fine-grained actions with a detailed description. This paper aims to address the challenge by proposing a novel hierarchical reinforcement learning framework for video captioning, where a high-level Manager module learns to design sub-goals and a low-level Worker module recognizes the primitive actions to fulfill the sub-goal. With this compositional framework to reinforce video captioning at different levels, our approach significantly outperforms all the baseline methods on a newly introduced large-scale dataset for fine-grained video captioning. Furthermore, our non-ensemble model has already achieved the state-of-the-art results on the widely-used MSR-VTT dataset.

CVPR 2018, with supplementary material

References in corpus (11)

• ADADELTA: An Adaptive Learning Rate Method
• Microsoft COCO Captions: Data Collection and Evaluation Server
• Improved Image Captioning via Policy Gradient optimization of SPIDEr
• An Actor-Critic Algorithm for Sequence Prediction
• Frame- and Segment-Level Features and Candidate Pool Evaluation for Video Caption Generation
• deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
• Hierarchical LSTM with Adjusted Temporal Attention for Video Captioning
• Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning
• Jointly Modeling Embedding and Translation to Bridge Video and Language
• Weakly Supervised Dense Video Captioning
• Semantic Compositional Networks for Visual Captioning

Cited by in corpus (7)

• Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
• Discriminability objective for training descriptive captions
• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
• An Attempt towards Interpretable Audio-Visual Video Captioning
• Detection and Description of Change in Visual Streams
• Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning
• Towards Diverse Paragraph Captioning for Untrimmed Videos