Multi-granularity Generator for Temporal Action Proposal
arXiv:1811.11524
Abstract
Temporal action proposal generation is an important task, aiming to localize the video segments containing human actions in an untrimmed video. In this paper, we propose a multi-granularity generator (MGG) to perform the temporal action proposal from different granularity perspectives, relying on the video visual features equipped with the position embedding information. First, we propose to use a bilinear matching model to exploit the rich local information within the video sequence. Afterwards, two components, namely segment proposal producer (SPP) and frame actionness producer (FAP), are combined to perform the task of temporal action proposal at two distinct granularities. SPP considers the whole video in the form of feature pyramid and generates segment proposals from one coarse perspective, while FAP carries out a finer actionness evaluation for each video frame. Our proposed MGG can be trained in an end-to-end fashion. By temporally adjusting the segment proposals with fine-grained frame actionness information, MGG achieves the superior performance over state-of-the-art methods on the public THUMOS-14 and ActivityNet-1.3 datasets. Moreover, we employ existing action classifiers to perform the classification of the proposals generated by MGG, leading to significant improvements compared against the competing methods for the video detection task.
arXiv commentAccepted to CVPR 2019
References in corpus (16)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Convolutional Sequence to Sequence Learning
- DSSD : Deconvolutional Single Shot Detector
- Describing Videos by Exploiting Temporal Structure
- Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs
- A Pursuit of Temporal Accuracy in General Activity Detection
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- BSN: Boundary Sensitive Network for Temporal Action Proposal Generation
- Temporal Context Network for Activity Localization in Videos
- TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals
- Temporal Convolution Based Action Proposal: Submission to ActivityNet 2017
- CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos
- Reconstruction Network for Video Captioning
- Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning
- UntrimmedNets for Weakly Supervised Action Recognition and Detection
- CTAP: Complementary Temporal Action Proposal Generation