Step-by-step Erasion, One-by-one Collection: A Weakly Supervised Temporal Action Detector

Abstract

Weakly supervised temporal action detection is a Herculean task in understanding untrimmed videos, since no supervisory signal except the video-level category label is available on training data. Under the supervision of category labels, weakly supervised detectors are usually built upon classifiers. However, there is an inherent contradiction between classifier and detector; i.e., a classifier in pursuit of high classification performance prefers top-level discriminative video clips that are extremely fragmentary, whereas a detector is obliged to discover the whole action instance without missing any relevant snippet. To reconcile this contradiction, we train a detector by driving a series of classifiers to find new actionness clips progressively, via step-by-step erasion from a complete video. During the test phase, all we need to do is to collect detection results from the one-by-one trained classifiers at various erasing steps. To assist in the collection process, a fully connected conditional random field is established to refine the temporal localization outputs. We evaluate our approach on two prevailing datasets, THUMOS'14 and ActivityNet. The experiments show that our detector advances state-of-the-art weakly supervised temporal action detection results, and even compares with quite a few strongly supervised methods.

To Appear in ACM Multimedia 2018

References in corpus (14)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors
• CNN: Single-label to Multi-label
• Temporal Action Detection with Structured Segment Networks
• A Pursuit of Temporal Accuracy in General Activity Detection
• Temporal Localization of Fine-Grained Actions in Videos by Domain Transfer from Web Images
• Temporal Context Network for Activity Localization in Videos
• TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals
• CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos
• Temporal Segment Networks for Action Recognition in Videos
• Monocular Depth Estimation using Multi-Scale Continuous CRFs as Sequential Deep Networks
• Connectionist Temporal Modeling for Weakly Supervised Action Labeling
• Searching Action Proposals via Spatial Actionness Estimation and Temporal Path Inference and Tracking

Cited by in corpus (8)

• Weakly-Supervised Action Localization by Generative Attention Modeling
• Hierarchical Deep Residual Reasoning for Temporal Moment Localization
• Action Unit Memory Network for Weakly Supervised Temporal Action Localization
• Scale Matters: Temporal Scale Aggregation Network for Precise Action Localization in Untrimmed Videos
• Foreground-Action Consistency Network for Weakly Supervised Temporal Action Localization
• Adversarial Seeded Sequence Growing for Weakly-Supervised Temporal Action Localization
• CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning
• The Blessings of Unlabeled Background in Untrimmed Videos