Action Recognition with Dynamic Image Networks
arXiv:1612.00738
Abstract
We introduce the concept of "dynamic image", a novel compact representation of videos useful for video analysis, particularly in combination with convolutional neural networks (CNNs). A dynamic image encodes temporal data such as RGB or optical flow videos by using the concept of `rank pooling'. The idea is to learn a ranking machine that captures the temporal evolution of the data and to use the parameters of the latter as a representation. When a linear ranking machine is used, the resulting representation is in the form of an image, which we call dynamic because it summarizes the video dynamics in addition of appearance. This is a powerful idea because it allows to convert any video to an image so that existing CNN models pre-trained for the analysis of still images can be immediately extended to videos. We also present an efficient and effective approximate rank pooling operator, accelerating standard rank pooling algorithms by orders of magnitude, and formulate that as a CNN layer. This new layer allows generalizing dynamic images to dynamic feature maps. We demonstrate the power of the new representations on standard benchmarks in action recognition achieving state-of-the-art performance.
14 pages, 9 figures, 9 tables
References in corpus (5)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Two-Stream Convolutional Networks for Action Recognition in Videos
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Caffe: Convolutional Architecture for Fast Feature Embedding
- On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization
Cited by in corpus (5)
- End-to-end Video-level Representation Learning for Action Recognition
- Action Recognition for Depth Video using Multi-view Dynamic Images
- NeXtVLAD: An Efficient Neural Network to Aggregate Frame-level Features for Large-scale Video Classification
- Discriminatively Learned Hierarchical Rank Pooling Networks
- Unsupervised Human Action Detection by Action Matching