Model-based Hand Pose Estimation for Generalized Hand Shape with Appearance Normalization

Abstract

Since the emergence of large annotated datasets, state-of-the-art hand pose estimation methods have been mostly based on discriminative learning. Recently, a hybrid approach has embedded a kinematic layer into the deep learning structure in such a way that the pose estimates obey the physical constraints of human hand kinematics. However, the existing approach relies on a single person's hand shape parameters, which are fixed constants. Therefore, the existing hybrid method has problems to generalize to new, unseen hands. In this work, we extend the kinematic layer to make the hand shape parameters learnable. In this way, the learnt network can generalize towards arbitrary hand shapes. Furthermore, inspired by the idea of Spatial Transformer Networks, we apply a cascade of appearance normalization networks to decrease the variance in the input data. The input images are shifted, rotated, and globally scaled to a similar appearance. The effectiveness and limitations of our proposed approach are extensively evaluated on the Hands 2017 challenge dataset and the NYU dataset.

References in corpus (15)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Hands Deep in Deep Learning for Hand Pose Estimation
• Fast and Robust Hand Tracking Using Detection-Guided Optimization
• Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation
• Model-based Deep Hand Pose Estimation
• Hand3D: Hand Pose Estimation using 3D Neural Network
• Compositional Human Pose Regression
• The 2017 Hands in the Million Challenge on 3D Hand Pose Estimation
• Towards Good Practices for Deep 3D Hand Pose Estimation
• End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth Data
• V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map
• BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis
• First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations
• Occlusion-aware Hand Pose Estimation Using Hierarchical Mixture Density Network
• Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals

Cited by in corpus (3)

• Fast and Accurate 3D Hand Pose Estimation via Recurrent Neural Network for Capturing Hand Articulations
• Disentangling Latent Hands for Image Synthesis and Pose Estimation
• Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation