Query-Conditioned Three-Player Adversarial Network for Video Summarization

Abstract

Video summarization plays an important role in video understanding by selecting key frames/shots. Traditionally, it aims to find the most representative and diverse contents in a video as short summaries. Recently, a more generalized task, query-conditioned video summarization, has been introduced, which takes user queries into consideration to learn more user-oriented summaries. In this paper, we propose a query-conditioned three-player generative adversarial network to tackle this challenge. The generator learns the joint representation of the user query and the video content, and the discriminator takes three pairs of query-conditioned summaries as the input to discriminate the real summary from a generated and a random one. A three-player loss is introduced for joint training of the generator and the discriminator, which forces the generator to learn better summary results, and avoids the generation of random trivial summaries. Experiments on a recently proposed query-conditioned video summarization benchmark dataset show the efficiency and efficacy of our proposed method.

arXiv comment13 pages, 3 figures, BMVC 2018

References in corpus (6)

• Video Summarization with Long Short-term Memory
• Summary Transfer: Exemplar-based Subset Selection for Video Summarization
• Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approach
• Semantic Video Trailers
• Query-Aware Sparse Coding for Multi-Video Summarization
• Unsupervised Object-Level Video Summarization with Online Motion Auto-Encoder

Cited by in corpus (5)

• Convolutional Hierarchical Attention Network for Query-Focused Video Summarization
• Image Conditioned Keyframe-Based Video Summarization Using Object Detection
• Query-controllable Video Summarization
• Compare and Select: Video Summarization with Multi-Agent Reinforcement Learning
• DeepQAMVS: Query-Aware Hierarchical Pointer Networks for Multi-Video Summarization