AutoLoc: Weakly-supervised Temporal Action Localization
arXiv:1807.08333
Abstract
Temporal Action Localization (TAL) in untrimmed video is important for many applications. But it is very expensive to annotate the segment-level ground truth (action class and temporal boundary). This raises the interest of addressing TAL with weak supervision, namely only video-level annotations are available during training). However, the state-of-the-art weakly-supervised TAL methods only focus on generating good Class Activation Sequence (CAS) over time but conduct simple thresholding on CAS to localize actions. In this paper, we first develop a novel weakly-supervised TAL framework called AutoLoc to directly predict the temporal boundary of each action instance. We propose a novel Outer-Inner-Contrastive (OIC) loss to automatically discover the needed segment-level supervision for training such a boundary predictor. Our method achieves dramatically improved performance: under the IoU threshold 0.5, our method improves mAP on THUMOS'14 from 13.7% to 21.2% and mAP on ActivityNet from 7.4% to 27.3%. It is also very encouraging to see that our weakly-supervised method achieves comparable results with some fully-supervised methods.
arXiv commentAccepted by ECCV'18
References in corpus (15)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- ConvNet Architecture Search for Spatiotemporal Feature Learning
- Temporal Action Detection with Structured Segment Networks
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- Advances in Human Action Recognition: A Survey
- TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals
- Cascaded Boundary Regression for Temporal Action Detection
- Review of Action Recognition and Detection Methods
- ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization
- Two-Phase Learning for Weakly Supervised Object Localization
- Predictive-Corrective Networks for Action Detection
- Weakly Supervised Semantic Segmentation using Web-Crawled Videos
- Deep Self-Taught Learning for Weakly Supervised Object Localization
- Weakly Supervised Dense Video Captioning
- Soft Proposal Networks for Weakly Supervised Object Localization