Temporal Action Localization using Long Short-Term Dependency

Abstract

Temporal action localization in untrimmed videos is an important but difficult task. Difficulties are encountered in the application of existing methods when modeling temporal structures of videos. In the present study, we developed a novel method, referred to as Gemini Network, for effective modeling of temporal structures and achieving high-performance temporal action localization. The significant improvements afforded by the proposed method are attributable to three major factors. First, the developed network utilizes two subnets for effective modeling of temporal structures. Second, three parallel feature extraction pipelines are used to prevent interference between the extractions of different stage features. Third, the proposed method utilizes auxiliary supervision, with the auxiliary classifier losses affording additional constraints for improving the modeling capability of the network. As a demonstration of its effectiveness, the Gemini Network was used to achieve state-of-the-art temporal action localization performance on two challenging datasets, namely, THUMOS14 and ActivityNet.

arXiv comment12pages, Trans

References in corpus (9)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• A Pursuit of Temporal Accuracy in General Activity Detection
• Untrimmed Video Classification for Activity Detection: submission to ActivityNet Challenge
• BSN: Boundary Sensitive Network for Temporal Action Proposal Generation
• Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
• TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals
• Cascaded Boundary Regression for Temporal Action Detection
• W-TALC: Weakly-supervised Temporal Activity Localization and Classification