View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data

Abstract

Skeleton-based human action recognition has recently attracted increasing attention due to the popularity of 3D skeleton data. One main challenge lies in the large view variations in captured human actions. We propose a novel view adaptation scheme to automatically regulate observation viewpoints during the occurrence of an action. Rather than re-positioning the skeletons based on a human defined prior criterion, we design a view adaptive recurrent neural network (RNN) with LSTM architecture, which enables the network itself to adapt to the most suitable observation viewpoints from end to end. Extensive experiment analyses show that the proposed view adaptive RNN model strives to (1) transform the skeletons of various views to much more consistent viewpoints and (2) maintain the continuity of the action rather than transforming every frame to the same position with the same body orientation. Our model achieves significant improvement over the state-of-the-art approaches on three benchmark datasets.

ICCV2017

References in corpus (2)

• Sequence to Sequence Learning with Neural Networks
• Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition

Cited by in corpus (17)

• SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition
• An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition
• Unsupervised Learning of View-invariant Action Representations
• Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• RGB-D-based Human Motion Recognition with Deep Learning: A Survey
• Action Machine: Rethinking Action Recognition in Trimmed Videos
• Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching
• View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition
• Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition
• Exploiting the ConvLSTM: Human Action Recognition using Raw Depth Video-Based Recurrent Neural Networks
• Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• View Invariant 3D Human Pose Estimation
• Unsupervised Feature Learning of Human Actions as Trajectories in Pose Embedding Manifold
• STS Classification with Dual-stream CNN
• Learning Chebyshev Basis in Graph Convolutional Networks for Skeleton-based Action Recognition
• Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition