Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals

Abstract

In this paper, we strive to answer two questions: What is the current state of 3D hand pose estimation from depth images? And, what are the next challenges that need to be tackled? Following the successful Hands In the Million Challenge (HIM2017), we investigate the top 10 state-of-the-art methods on three tasks: single frame 3D pose estimation, 3D hand tracking, and hand pose estimation during object interaction. We analyze the performance of different CNN structures with regard to hand shape, joint visibility, view point and articulation distributions. Our findings include: (1) isolated 3D hand pose estimation achieves low mean errors (10 mm) in the view point range of [70, 120] degrees, but it is far from being solved for extreme view points; (2) 3D volumetric representations outperform 2D CNNs, better capturing the spatial structure of the depth data; (3) Discriminative methods still generalize poorly to unseen hand shapes; (4) While joint occlusions pose a challenge for most methods, explicit modeling of structure constraints can significantly narrow the gap between errors on visible and occluded joints.

References in corpus (13)

• Deep Residual Learning for Image Recognition
• Hands Deep in Deep Learning for Hand Pose Estimation
• Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
• Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation
• 3D Hand Pose Tracking and Estimation Using Stereo Matching
• Hand3D: Hand Pose Estimation using 3D Neural Network
• Compositional Human Pose Regression
• The 2017 Hands in the Million Challenge on 3D Hand Pose Estimation
• Towards Good Practices for Deep 3D Hand Pose Estimation
• End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth Data
• Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
• BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis
• Depth-based hand pose estimation: methods, data, and challenges

Cited by in corpus (4)

• Generating Realistic Training Images Based on Tonality-Alignment Generative Adversarial Networks for Hand Pose Estimation
• Model-based Hand Pose Estimation for Generalized Hand Shape with Appearance Normalization
• DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth
• StarMap for Category-Agnostic Keypoint and Viewpoint Estimation