Hand3D: Hand Pose Estimation using 3D Neural Network

Abstract

We propose a novel 3D neural network architecture for 3D hand pose estimation from a single depth image. Different from previous works that mostly run on 2D depth image domain and require intermediate or post process to bring in the supervision from 3D space, we convert the depth map to a 3D volumetric representation, and feed it into a 3D convolutional neural network(CNN) to directly produce the pose in 3D requiring no further process. Our system does not require the ground truth reference point for initialization, and our network architecture naturally integrates both local feature and global context in 3D space. To increase the coverage of the hand pose space of the training data, we render synthetic depth image by transferring hand pose from existing real image datasets. We evaluation our algorithm on two public benchmarks and achieve the state-of-the-art performance. The synthetic hand pose dataset will be available.

References in corpus (3)

• Hands Deep in Deep Learning for Hand Pose Estimation
• Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNs
• DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding

Cited by in corpus (14)

• V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map
• A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Image
• ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation
• Context-Aware Deep Spatio-Temporal Network for Hand Pose Estimation from Depth Images
• Simultaneous Hand Pose and Skeleton Bone-Lengths Estimation from a Single Depth Image
• Model-based Hand Pose Estimation for Generalized Hand Shape with Appearance Normalization
• DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth
• Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
• Silhouette-Net: 3D Hand Pose Estimation from Silhouettes
• Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layer
• Temporal-Aware Self-Supervised Learning for 3D Hand Pose and Mesh Estimation in Videos
• HMTNet:3D Hand Pose Estimation from Single Depth Image Based on Hand Morphological Topology
• Robust and customized methods for real-time hand gesture recognition under object-occlusion
• DGGAN: Depth-image Guided Generative Adversarial Networks for Disentangling RGB and Depth Images in 3D Hand Pose Estimation