ConvNet Architecture Search for Spatiotemporal Feature Learning

Abstract

Learning image representations with ConvNets by pre-training on ImageNet has proven useful across many visual understanding tasks including object detection, semantic segmentation, and image captioning. Although any image representation can be applied to video frames, a dedicated spatiotemporal representation is still vital in order to incorporate motion patterns that cannot be captured by appearance based models alone. This paper presents an empirical ConvNet architecture search for spatiotemporal feature learning, culminating in a deep 3-dimensional (3D) Residual ConvNet. Our proposed architecture outperforms C3D by a good margin on Sports-1M, UCF101, HMDB51, THUMOS14, and ASLAN while being 2 times faster at inference time, 2 times smaller in model size, and having a more compact representation.

Cited by in corpus (8)

• Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification
• BMN: Boundary-Matching Network for Temporal Action Proposal Generation
• DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition
• Learning Video Representations from Correspondence Proposals
• Towards Real-Time Action Recognition on Mobile Devices Using Deep Models
• IF-TTN: Information Fused Temporal Transformation Network for Video Action Recognition
• LP-3DCNN: Unveiling Local Phase in 3D Convolutional Neural Networks
• AVD: Adversarial Video Distillation