Revisiting Anchor Mechanisms for Temporal Action Localization

Abstract

Most of the current action localization methods follow an anchor-based pipeline: depicting action instances by pre-defined anchors, learning to select the anchors closest to the ground truth, and predicting the confidence of anchors with refinements. Pre-defined anchors set prior about the location and duration for action instances, which facilitates the localization for common action instances but limits the flexibility for tackling action instances with drastic varieties, especially for extremely short or extremely long ones. To address this problem, this paper proposes a novel anchor-free action localization module that assists action localization by temporal points. Specifically, this module represents an action instance as a point with its distances to the starting boundary and ending boundary, alleviating the pre-defined anchor restrictions in terms of action localization and duration. The proposed anchor-free module is capable of predicting the action instances whose duration is either extremely short or extremely long. By combining the proposed anchor-free module with a conventional anchor-based module, we propose a novel action localization framework, called A2Net. The cooperation between anchor-free and anchor-based modules achieves superior performance to the state-of-the-art on THUMOS14 (45.5% vs. 42.8%). Furthermore, comprehensive experiments demonstrate the complementarity between the anchor-free and the anchor-based module, making A2Net simple but effective.

Accept by TIP, code: https://github.com/LeYangNwpu/A2Net

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Towards Good Practices for Very Deep Two-Stream ConvNets
• Single Shot Temporal Action Detection
• A Pursuit of Temporal Accuracy in General Activity Detection
• CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016
• ActivityNet Challenge 2017 Summary
• SF-Net: Single-Frame Supervision for Temporal Action Localization
• Deep Concept-wise Temporal Convolutional Networks for Action Localization

Cited by in corpus (12)

• End-to-end Temporal Action Detection with Transformer
• Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos
• Learning Salient Boundary Feature for Anchor-free Temporal Action Localization
• RGB Stream Is Enough for Temporal Action Detection
• Occluded Gait Recognition with Mixture of Experts: An Action Detection Perspective
• Point-Level Temporal Action Localization: Bridging Fully-supervised Proposals to Weakly-supervised Losses
• Privileged Knowledge Distillation for Online Action Detection
• Towards High-Quality Temporal Action Detection with Sparse Proposals
• Speech-Forensics: Towards Comprehensive Synthetic Speech Dataset Establishment and Analysis
• HateClipSeg: A Segment-Level Annotated Dataset for Fine-Grained Hate Video Detection
• The 6th AI City Challenge
• Background-Click Supervision for Temporal Action Localization