S3D: Single Shot multi-Span Detector via Fully 3D Convolutional Networks
arXiv:1807.08069
Abstract
In this paper, we present a novel Single Shot multi-Span Detector for temporal activity detection in long, untrimmed videos using a simple end-to-end fully three-dimensional convolutional (Conv3D) network. Our architecture, named S3D, encodes the entire video stream and discretizes the output space of temporal activity spans into a set of default spans over different temporal locations and scales. At prediction time, S3D predicts scores for the presence of activity categories in each default span and produces temporal adjustments relative to the span location to predict the precise activity duration. Unlike many state-of-the-art systems that require a separate proposal and classification stage, our S3D is intrinsically simple and dedicatedly designed for single-shot, end-to-end temporal activity detection. When evaluating on THUMOS'14 detection benchmark, S3D achieves state-of-the-art performance and is very efficient and can operate at 1271 FPS.
BMVC 2018 Oral
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Towards Good Practices for Very Deep Two-Stream ConvNets
- Convolutional Two-Stream Network Fusion for Video Action Recognition
- Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs
- Temporal Action Detection with Structured Segment Networks
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- Deep Reinforcement Learning for Visual Object Tracking in Videos
- StyleBank: An Explicit Representation for Neural Image Style Transfer
- Temporal Context Network for Activity Localization in Videos
- Coherent Online Video Style Transfer
Cited by in corpus (15)
- CLIP2Video: Mastering Video-Text Retrieval via Image CLIP
- MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment
- Weakly-Supervised Action Localization by Generative Attention Modeling
- RGB Stream Is Enough for Temporal Action Detection
- Facial Tic Detection in Untrimmed Videos of Tourette Syndrome Patients
- Dynamic Temporal Pyramid Network: A Closer Look at Multi-Scale Modeling for Activity Detection
- Learning Sparse 2D Temporal Adjacent Networks for Temporal Action Localization
- Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
- Learning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning
- Scale Matters: Temporal Scale Aggregation Network for Precise Action Localization in Untrimmed Videos
- AFO-TAD: Anchor-free One-Stage Detector for Temporal Action Detection
- CMSN: Continuous Multi-stage Network and Variable Margin Cosine Loss for Temporal Action Proposal Generation
- TAN: Temporal Aggregation Network for Dense Multi-label Action Recognition
- Deep Point-wise Prediction for Action Temporal Proposal
- Finding Action Tubes with a Sparse-to-Dense Framework