WSLLN: Weakly Supervised Natural Language Localization Networks
arXiv:1909.00239
Abstract
We propose weakly supervised language localization networks (WSLLN) to detect events in long, untrimmed videos given language queries. To learn the correspondence between visual segments and texts, most previous methods require temporal coordinates (start and end times) of events for training, which leads to high costs of annotation. WSLLN relieves the annotation burden by training with only video-sentence pairs without accessing to temporal locations of events. With a simple end-to-end structure, WSLLN measures segment-text consistency and conducts segment selection (conditioned on the text) simultaneously. Results from both are merged and optimized as a video-sentence matching problem. Experiments on ActivityNet Captions and DiDeMo demonstrate that WSLLN achieves state-of-the-art performance.
arXiv commentaccepted by EMNLP2019
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Semi-Supervised Classification with Graph Convolutional Networks
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Weakly Supervised Dense Event Captioning in Videos
- MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment
- To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regression
- Temporal Recurrent Networks for Online Action Detection
- StartNet: Online Detection of Action Start in Untrimmed Videos
Cited by in corpus (4)
- A Closer Look at Temporal Sentence Grounding in Videos: Dataset and Metric
- Weak Supervision and Referring Attention for Temporal-Textual Association Learning
- Reinforcement Learning for Weakly Supervised Temporal Grounding of Natural Language in Untrimmed Videos
- A Survey on Natural Language Video Localization