Depth2Action: Exploring Embedded Depth for Large-Scale Action Recognition

Abstract

This paper performs the first investigation into depth for large-scale human action recognition in video where the depth cues are estimated from the videos themselves. We develop a new framework called depth2action and experiment thoroughly into how best to incorporate the depth information. We introduce spatio-temporal depth normalization (STDN) to enforce temporal consistency in our estimated depth sequences. We also propose modified depth motion maps (MDMM) to capture the subtle temporal changes in depth. These two components significantly improve the action recognition performance. We evaluate our depth2action framework on three large-scale action recognition video benchmarks. Our model achieves state-of-the-art performance when combined with appearance and motion information thus demonstrating that depth2action is indeed complementary to existing approaches.

ECCVW 2016, Web-scale Vision and Social Media (VSM) workshop

References in corpus (9)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Unsupervised Learning of Video Representations using LSTMs
• Towards Good Practices for Very Deep Two-Stream ConvNets
• DAP3D-Net: Where, What and How Actions Occur in Videos?
• Improving Human Activity Recognition Through Ranking and Re-ranking
• Do Less and Achieve More: Training CNNs for Action Recognition Utilizing Action Images from the Web
• Structured Depth Prediction in Challenging Monocular Video Sequences

Cited by in corpus (11)

• Multi-Modality Fusion based on Consensus-Voting and 3D Convolution for Isolated Gesture Recognition
• Guided Optical Flow Learning
• UntrimmedNets for Weakly Supervised Action Recognition and Detection
• Combined Static and Motion Features for Deep-Networks Based Activity Recognition in Videos
• Towards Universal Representation for Unseen Action Recognition
• Deep Texture Manifold for Ground Terrain Recognition
• Fine-Grained Land Use Classification at the City Scale Using Ground-Level Images
• Learning Optical Flow via Dilated Networks and Occlusion Reasoning
• Random Temporal Skipping for Multirate Video Analysis
• Efficient Action Detection in Untrimmed Videos via Multi-Task Learning
• Large-Scale Mapping of Human Activity using Geo-Tagged Videos