Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling
arXiv:2106.11811
Abstract
Weakly-Supervised Temporal Action Localization (WS-TAL) task aims to recognize and localize temporal starts and ends of action instances in an untrimmed video with only video-level label supervision. Due to lack of negative samples of background category, it is difficult for the network to separate foreground and background, resulting in poor detection performance. In this report, we present our 2021 HACS Challenge - Weakly-supervised Learning Track solution that based on BaSNet to address above problem. Specifically, we first adopt pre-trained CSN, Slowfast, TDN, and ViViT as feature extractors to get feature sequences. Then our proposed Local-Global Background Modeling Network (LGBM-Net) is trained to localize instances by using only video-level labels based on Multi-Instance Learning (MIL). Finally, we ensemble multiple models to get the final detection results and reach 22.45% mAP on the test set
arXiv commentCVPR-2021 HACS Challenge - Weakly-supervised Learning Track champion solution (1st Place)
References in corpus (9)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- LSTM: A Search Space Odyssey
- A Short Note on the Kinetics-700-2020 Human Action Dataset
- Weakly Supervised Action Localization by Sparse Temporal Pooling Network
- TDN: Temporal Difference Networks for Efficient Action Recognition
- Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance Learning
- Temporal Context Aggregation Network for Temporal Action Proposal Refinement
- Self-Supervised Learning for Semi-Supervised Temporal Action Proposal
- Self-supervised Motion Learning from Static Images