LAP-Net: Adaptive Features Sampling via Learning Action Progression for Online Action Detection

Abstract

Online action detection is a task with the aim of identifying ongoing actions from streaming videos without any side information or access to future frames. Recent methods proposed to aggregate fixed temporal ranges of invisible but anticipated future frames representations as supplementary features and achieved promising performance. They are based on the observation that human beings often detect ongoing actions by contemplating the future vision simultaneously. However, we observed that at different action progressions, the optimal supplementary features should be obtained from distinct temporal ranges instead of simply fixed future temporal ranges. To this end, we introduce an adaptive features sampling strategy to overcome the mentioned variable-ranges of optimal supplementary features. Specifically, in this paper, we propose a novel Learning Action Progression Network termed LAP-Net, which integrates an adaptive features sampling strategy. At each time step, this sampling strategy first estimates current action progression and then decide what temporal ranges should be used to aggregate the optimal supplementary features. We evaluated our LAP-Net on three benchmark datasets, TVSeries, THUMOS-14 and HDD. The extensive experiments demonstrate that with our adaptive feature sampling strategy, the proposed LAP-Net can significantly outperform current state-of-the-art methods with a large margin.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Categorical Reparameterization with Gumbel-Softmax
• Convolutional Two-Stream Network Fusion for Video Action Recognition
• R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
• Temporal Context Network for Activity Localization in Videos
• SkipNet: Learning Dynamic Routing in Convolutional Networks
• End-to-end Learning of Action Detection from Frame Glimpses in Videos
• Encouraging LSTMs to Anticipate Actions Very Early
• Temporal Recurrent Networks for Online Action Detection
• AdaFrame: Adaptive Frame Selection for Fast Video Recognition
• Dynamic Zoom-in Network for Fast Object Detection in Large Images
• Learning 3D-aware Egocentric Spatial-Temporal Interaction via Graph Convolutional Networks
• Learning to Discriminate Information for Online Action Detection

Cited by in corpus (1)

• Long Short-Term Transformer for Online Action Detection