Exploring Stronger Feature for Temporal Action Localization
arXiv:2106.13014
Abstract
Temporal action localization aims to localize starting and ending time with action category. Limited by GPU memory, mainstream methods pre-extract features for each video. Therefore, feature quality determines the upper bound of detection performance. In this technical report, we explored classic convolution-based backbones and the recent surge of transformer-based backbones. We found that the transformer-based methods can achieve better classification performance than convolution-based, but they cannot generate accuracy action proposals. In addition, extracting features with larger frame resolution to reduce the loss of spatial information can also effectively improve the performance of temporal action localization. Finally, we achieve 42.42% in terms of mAP on validation set with a single SlowFast feature by a simple combination: BMN+TCANet, which is 1.87% higher than the result of 2020's multi-model ensemble. Finally, we achieve Rank 1st on the CVPR2021 HACS supervised Temporal Action Localization Challenge.
arXiv commentRank 1st on the CVPR2021 HACS supervised Temporal Action Localization Challenge
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- Is Space-Time Attention All You Need for Video Understanding?
- A Short Note on the Kinetics-700-2020 Human Action Dataset
- A Short Note about Kinetics-600
- Self-supervised Co-training for Video Representation Learning
- Temporal Context Aggregation Network for Temporal Action Proposal Refinement
- Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition
- Self-Supervised Learning for Semi-Supervised Temporal Action Proposal
- Self-supervised Motion Learning from Static Images
- CBR-Net: Cascade Boundary Refinement Network for Action Detection: Submission to ActivityNet Challenge 2020 (Task 1)
- Temporal Fusion Network for Temporal Action Localization:Submission to ActivityNet Challenge 2020 (Task E)
- Augmenting Proposals by the Detector Itself