Hierarchical Attention Network for Action Recognition in Videos

Abstract

Understanding human actions in wild videos is an important task with a broad range of applications. In this paper we propose a novel approach named Hierarchical Attention Network (HAN), which enables to incorporate static spatial information, short-term motion information and long-term video temporal structures for complex human action understanding. Compared to recent convolutional neural network based approaches, HAN has following advantages (1) HAN can efficiently capture video temporal structures in a longer range; (2) HAN is able to reveal temporal transitions between frame chunks with different time steps, i.e. it explicitly models the temporal transitions between frames as well as video segments and (3) with a multiple step spatial temporal attention mechanism, HAN automatically learns important regions in video frames and temporal segments in the video. The proposed model is trained and evaluated on the standard video action benchmarks, i.e., UCF-101 and HMDB-51, and it significantly outperforms the state-of-the arts

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ADADELTA: An Adaptive Learning Rate Method
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Recurrent Neural Network Regularization
• DRAW: A Recurrent Neural Network For Image Generation
• Learning to Execute

Cited by in corpus (13)

• A General Survey on Attention Mechanisms in Deep Learning
• A Deep Learning Framework for Assessing Physical Rehabilitation Exercises
• SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction
• Video-based Human Action Recognition using Deep Learning: A Review
• Knowing What, Where and When to Look: Efficient Video Action Modeling with Attention
• Learning Representative Temporal Features for Action Recognition
• Interpretable Spatio-temporal Attention for Video Action Recognition
• Adding Attentiveness to the Neurons in Recurrent Neural Networks
• Relational Long Short-Term Memory for Video Action Recognition
• Hierarchical Multi-scale Attention Networks for Action Recognition
• Temporal-Spatial Mapping for Action Recognition
• DAVID: Dual-Attentional Video Deblurring
• Learning Comprehensive Motion Representation for Action Recognition