Dense Regression Network for Video Grounding

Abstract

We address the problem of video grounding from natural language queries. The key challenge in this task is that one training video might only contain a few annotated starting/ending frames that can be used as positive examples for model training. Most conventional approaches directly train a binary classifier using such imbalance data, thus achieving inferior results. The key idea of this paper is to use the distances between the frame within the ground truth and the starting (ending) frame as dense supervisions to improve the video grounding accuracy. Specifically, we design a novel dense regression network (DRN) to regress the distances from each frame to the starting (ending) frame of the video segment described by the query. We also propose a simple but effective IoU regression head module to explicitly consider the localization quality of the grounding results (i.e., the IoU between the predicted location and the ground truth). Experimental results show that our approach significantly outperforms state-of-the-arts on three datasets (i.e., Charades-STA, ActivityNet-Captions, and TACoS).

CVPR 2020

References in corpus (6)

• DenseBox: Unifying Landmark Localization with End to End Object Detection
• Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videos
• Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis
• ExCL: Extractive Clip Localization Using Natural Language Descriptions
• NAT: Neural Architecture Transformer for Accurate and Compact Architectures
• Multi-marginal Wasserstein GAN

Cited by in corpus (6)

• Visual Keyword Spotting with Attention
• A Survey on Natural Language Video Localization
• Towards Accurate Text-based Image Captioning with Content Diversity Exploration
• Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization
• On Pursuit of Designing Multi-modal Transformer for Video Grounding
• Deconfounded Video Moment Retrieval with Causal Intervention