Improving Sequential Determinantal Point Processes for Supervised Video Summarization
arXiv:1807.10957
Abstract
It is now much easier than ever before to produce videos. While the ubiquitous video data is a great source for information discovery and extraction, the computational challenges are unparalleled. Automatically summarizing the videos has become a substantial need for browsing, searching, and indexing visual content. This paper is in the vein of supervised video summarization using sequential determinantal point process (SeqDPP), which models diversity by a probabilistic distribution. We improve this model in two folds. In terms of learning, we propose a large-margin algorithm to address the exposure bias problem in SeqDPP. In terms of modeling, we design a new probabilistic distribution such that, when it is integrated into SeqDPP, the resulting model accepts user input about the expected length of the summary. Moreover, we also significantly extend a popular video summarization dataset by 1) more egocentric videos, 2) dense user annotations, and 3) a refined evaluation scheme. We conduct extensive experiments on this dataset (about 60 hours of videos in total) and compare our approach to several competitive baselines.
References in corpus (10)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Sequence Level Training with Recurrent Neural Networks
- Determinantal Processes and Independence
- Sequence to Sequence -- Video to Text
- Sequence-to-Sequence Learning as Beam-Search Optimization
- Video Summarization with Long Short-term Memory
- Minimum Risk Training for Neural Machine Translation
- VideoSET: Video Summary Evaluation through Text
- Summary Transfer: Exemplar-based Subset Selection for Video Summarization
- Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approach