The Best of Both Worlds: Combining Data-independent and Data-driven Approaches for Action Recognition
arXiv:1505.04427
Abstract
Motivated by the success of data-driven convolutional neural networks (CNNs) in object recognition on static images, researchers are working hard towards developing CNN equivalents for learning video features. However, learning video features globally has proven to be quite a challenge due to its high dimensionality, the lack of labelled data and the difficulty in processing large-scale video data. Therefore, we propose to leverage effective techniques from both data-driven and data-independent approaches to improve action recognition system. Our contribution is three-fold. First, we propose a two-stream Stacked Convolutional Independent Subspace Analysis (ConvISA) architecture to show that unsupervised learning methods can significantly boost the performance of traditional local features extracted from data-independent models. Second, we demonstrate that by learning on video volumes detected by Improved Dense Trajectory (IDT), we can seamlessly combine our novel local descriptors with hand-crafted descriptors. Thus we can utilize available feature enhancing techniques developed for hand-crafted descriptors. Finally, similar to multi-class classification framework in CNNs, we propose a training-free re-ranking technique that exploits the relationship among action classes to improve the overall performance. Our experimental results on four benchmark action recognition datasets show significantly improved performance.
References in corpus (7)
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- CNN Features off-the-shelf: an Astounding Baseline for Recognition
- A note on the group lasso and a sparse group lasso
- Video (language) modeling: a baseline for generative models of natural videos
- Beyond Short Snippets: Deep Networks for Video Classification
- Bag of Visual Words and Fusion Methods for Action Recognition: Comprehensive Study and Good Practice
- Feature sampling and partitioning for visual vocabulary generation on large action classification datasets