Weakly Supervised Dense Event Captioning in Videos
arXiv:1812.03849
Abstract
Dense event captioning aims to detect and describe all events of interest contained in a video. Despite the advanced development in this area, existing methods tackle this task by making use of dense temporal annotations, which is dramatically source-consuming. This paper formulates a new problem: weakly supervised dense event captioning, which does not require temporal segment annotations for model training. Our solution is based on the one-to-one correspondence assumption, each caption describes one temporal segment, and each temporal segment has one caption, which holds in current benchmark datasets and most real-world cases. We decompose the problem into a pair of dual problems: event captioning and sentence localization and present a cycle system to train our model. Extensive experimental results are provided to demonstrate the ability of our model on both dense event captioning and sentence localization in videos.
arXiv commentNeurIPS 2018
Cited by in corpus (23)
- VLANet: Video-Language Alignment Network for Weakly-Supervised Video Moment Retrieval
- A Closer Look at Temporal Sentence Grounding in Videos: Dataset and Metric
- Look Closer to Ground Better: Weakly-Supervised Temporal Grounding of Sentence in Video
- Graph Convolutional Networks for Temporal Action Localization
- AI Choreographer: Music Conditioned 3D Dance Generation with AIST++
- To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regression
- Weakly-Supervised Video Moment Retrieval via Semantic Completion Network
- Reinforcement Learning for Weakly Supervised Temporal Grounding of Natural Language in Untrimmed Videos
- Cross-Sentence Temporal and Semantic Relations in Video Activity Localisation
- A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformer
- Conditional Extreme Value Theory for Open Set Video Domain Adaptation
- Regularized Two-Branch Proposal Networks for Weakly-Supervised Moment Retrieval in Videos
- Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Grounding
- Towards Debiasing Temporal Sentence Grounding in Video
- A Survey on Natural Language Video Localization
- WSLLN: Weakly Supervised Natural Language Localization Networks
- Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language
- A Survey on Temporal Sentence Grounding in Videos
- Self-supervised Learning for Semi-supervised Temporal Language Grounding
- Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching
- Multi-modal Dense Video Captioning
- Syntax Customized Video Captioning by Imitating Exemplar Sentences
- Multi-scale 2D Representation Learning for weakly-supervised moment retrieval