Consensus-based Sequence Training for Video Captioning
arXiv:1712.09532
Abstract
Captioning models are typically trained using the cross-entropy loss. However, their performance is evaluated on other metrics designed to better correlate with human assessments. Recently, it has been shown that reinforcement learning (RL) can directly optimize these metrics in tasks such as captioning. However, this is computationally costly and requires specifying a baseline reward at each step to make training converge. We propose a fast approach to optimize one's objective of interest through the REINFORCE algorithm. First we show that, by replacing model samples with ground-truth sentences, RL training can be seen as a form of weighted cross-entropy loss, giving a fast, RL-based pre-training algorithm. Second, we propose to use the consensus among ground-truth captions of the same video as the baseline reward. This can be computed very efficiently. We call the complete proposal Consensus-based Sequence Training (CST). Applied to the MSRVTT video captioning benchmark, our proposals train significantly faster than comparable methods and establish a new state-of-the-art on the task, improving the CIDEr score from 47.3 to 54.2.
11 pages, 4 figures, 5 tables. Github repo at https://github.com/mynlp/cst_captioning
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Deep Residual Learning for Image Recognition
- Recurrent Neural Network Regularization
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Improved Image Captioning via Policy Gradient optimization of SPIDEr
- A note on the evaluation of generative models
- Professor Forcing: A New Algorithm for Training Recurrent Networks
- Learning Spatiotemporal Features with 3D Convolutional Networks
- Frame- and Segment-Level Features and Candidate Pool Evaluation for Video Caption Generation
- Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
- Generating Visual Explanations
- CIDEr: Consensus-based Image Description Evaluation
- Weakly Supervised Dense Video Captioning