Res3ATN -- Deep 3D Residual Attention Network for Hand Gesture Recognition in Videos

Abstract

Hand gesture recognition is a strenuous task to solve in videos. In this paper, we use a 3D residual attention network which is trained end to end for hand gesture recognition. Based on the stacked multiple attention blocks, we build a 3D network which generates different features at each attention block. Our 3D attention based residual network (Res3ATN) can be built and extended to very deep layers. Using this network, an extensive analysis is performed on other 3D networks based on three publicly available datasets. The Res3ATN network performance is compared to C3D, ResNet-10, and ResNext-101 networks. We also study and evaluate our baseline network with different number of attention blocks. The comparison shows that the 3D residual attention network with 3 attention blocks is robust in attention learning and is able to classify the gestures with better accuracy, thus outperforming existing networks.

arXiv comment10 pages, 4 figures, International Conference on 3D Vision (3DV 2019), Quebec City, Canada, September 16-19, 2019

References in corpus (11)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Attention-Based Models for Speech Recognition
• Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors
• DRAW: A Recurrent Neural Network For Image Generation
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• LSTM-based Deep Learning Models for Non-factoid Answer Selection
• Attention is All We Need: Nailing Down Object-centric Attention for Egocentric Activity Recognition
• Real-time Hand Gesture Detection and Classification Using Convolutional Neural Networks
• Deep video gesture recognition using illumination invariants

Cited by in corpus (5)

• Multiscaled Multi-Head Attention-based Video Transformer Network for Hand Gesture Recognition
• HeadPosr: End-to-end Trainable Head Pose Estimation using Transformer Encoders
• Weakly Supervised Attention Model for RV StrainClassification from volumetric CTPA Scans
• ASL Recognition with Metric-Learning based Lightweight Network
• 3D attention mechanism for fine-grained classification of table tennis strokes using a Twin Spatio-Temporal Convolutional Neural Networks