Semantic Adversarial Network with Multi-scale Pyramid Attention for Video Classification
arXiv:1903.02155
Abstract
Two-stream architecture have shown strong performance in video classification task. The key idea is to learn spatio-temporal features by fusing convolutional networks spatially and temporally. However, there are some problems within such architecture. First, it relies on optical flow to model temporal information, which are often expensive to compute and store. Second, it has limited ability to capture details and local context information for video data. Third, it lacks explicit semantic guidance that greatly decrease the classification performance. In this paper, we proposed a new two-stream based deep framework for video classification to discover spatial and temporal information only from RGB frames, moreover, the multi-scale pyramid attention (MPA) layer and the semantic adversarial learning (SAL) module is introduced and integrated in our framework. The MPA enables the network capturing global and local feature to generate a comprehensive representation for video, and the SAL can make this representation gradually approximate to the real video semantics in an adversarial manner. Experimental results on two public benchmarks demonstrate our proposed methods achieves state-of-the-art results on standard video datasets.
References in corpus (6)
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Convolutional Two-Stream Network Fusion for Video Action Recognition
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
- Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
- Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval
- Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning