Action-Attending Graphic Neural Network
arXiv:1711.06427 · doi:10.1109/TIP.2018.2815744
Abstract
The motion analysis of human skeletons is crucial for human action recognition, which is one of the most active topics in computer vision. In this paper, we propose a fully end-to-end action-attending graphic neural network (AGNN) for skeleton-based action recognition, in which each irregular skeleton is structured as an undirected attribute graph. To extract high-level semantic representation from skeletons, we perform the local spectral graph filtering on the constructed attribute graphs like the standard image convolution operation. Considering not all joints are informative for action analysis, we design an action-attending layer to detect those salient action units (AUs) by adaptively weighting skeletal joints. Herein the filtering responses are parameterized into a weighting function irrelevant to the order of input nodes. To further encode continuous motion variations, the deep features learnt from skeletal graphs are gathered along consecutive temporal slices and then fed into a recurrent gated network. Finally, the spectral graph filtering, action-attending and recurrent temporal encoding are integrated together to jointly train for the sake of robust action recognition as well as the intelligibility of human actions. To evaluate our AGNN, we conduct extensive experiments on four benchmark skeleton-based action datasets, including the large-scale challenging NTU RGB+D dataset. The experimental results demonstrate that our network achieves the state-of-the-art performances.
References in corpus (13)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Sequence to Sequence Learning with Neural Networks
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
- Spectral Networks and Locally Connected Networks on Graphs
- Attention-Based Models for Speech Recognition
- Unsupervised Learning of Video Representations using LSTMs
- Learning Convolutional Neural Networks for Graphs
- Unstructured Human Activity Detection from RGBD Images
- An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
- NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
- A Riemannian Network for SPD Matrix Learning
- Deep Learning on Lie Groups for Skeleton-based Action Recognition
Cited by in corpus (8)
- Context-Dependent Diffusion Network for Visual Relationship Detection
- Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos
- Human Action Performance using Deep Neuro-Fuzzy Recurrent Attention Model
- Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition
- Tensor graph convolutional neural network
- Action Recognition with Spatio-Temporal Visual Attention on Skeleton Image Sequences
- Gaussian-Induced Convolution for Graphs
- Skeleton-based Approaches based on Machine Vision: A Survey