Hallucinating Optical Flow Features for Video Classification
arXiv:1905.11799
Abstract
Appearance and motion are two key components to depict and characterize the video content. Currently, the two-stream models have achieved state-of-the-art performances on video classification. However, extracting motion information, specifically in the form of optical flow features, is extremely computationally expensive, especially for large-scale video classification. In this paper, we propose a motion hallucination network, namely MoNet, to imagine the optical flow features from the appearance features, with no reliance on the optical flow computation. Specifically, MoNet models the temporal relationships of the appearance features and exploits the contextual relationships of the optical flow features with concurrent connections. Extensive experimental results demonstrate that the proposed MoNet can effectively and efficiently hallucinate the optical flow features, which together with the appearance features consistently improve the video classification performances. Moreover, MoNet can help cutting down almost a half of computational and data-storage burdens for the two-stream video classification. Our code is available at: https://github.com/YongyiTang92/MoNet-Features.
Accepted by IJCAI 2019
References in corpus (7)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Neural Message Passing for Quantum Chemistry
- Conditional Random Fields as Recurrent Neural Networks
- YouTube-8M: A Large-Scale Video Classification Benchmark
- Learnable pooling with Context Gating for video classification
- Non-local NetVLAD Encoding for Video Classification
- NeXtVLAD: An Efficient Neural Network to Aggregate Frame-level Features for Large-scale Video Classification