3D Hand Pose Tracking and Estimation Using Stereo Matching

Abstract

3D hand pose tracking/estimation will be very important in the next generation of human-computer interaction. Most of the currently available algorithms rely on low-cost active depth sensors. However, these sensors can be easily interfered by other active sources and require relatively high power consumption. As a result, they are currently not suitable for outdoor environments and mobile devices. This paper aims at tracking/estimating hand poses using passive stereo which avoids these limitations. A benchmark with 18,000 stereo image pairs and 18,000 depth images captured from different scenarios and the ground-truth 3D positions of palm and finger joints (obtained from the manual label) is thus proposed. This paper demonstrates that the performance of the state-of-the art tracking/estimation algorithms can be maintained with most stereo matching algorithms on the proposed benchmark, as long as the hand segmentation is correct. As a result, a novel stereo-based hand segmentation algorithm specially designed for hand tracking/estimation is proposed. The quantitative evaluation demonstrates that the proposed algorithm is suitable for the state-of-the-art hand pose tracking/estimation algorithms and the tracking quality is comparable to the use of active depth sensors under different challenging scenarios.

References in corpus (2)

• Fast and Robust Hand Tracking Using Detection-Guided Optimization
• Depth-based hand pose estimation: methods, data, and challenges

Cited by in corpus (26)

• 3D Hand Pose and Shape Estimation from RGB Images for Keypoint-Based Hand Gesture Recognition
• FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integration
• 3D Hand Shape and Pose Estimation from a Single RGB Image
• Survey on Hand Gesture Recognition from Visual Input
• Monocular Total Capture: Posing Face, Body, and Hands in the Wild
• Generating Realistic Training Images Based on Tonality-Alignment Generative Adversarial Networks for Hand Pose Estimation
• RGB-based 3D Hand Pose Estimation via Privileged Learning with Depth Images
• End-to-end Hand Mesh Recovery from a Monocular RGB Image
• Disentangling Latent Hands for Image Synthesis and Pose Estimation
• GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB
• Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data
• MM-Hand: 3D-Aware Multi-Modal Guided Hand Generative Network for 3D Hand Pose Synthesis
• Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
• Back to RGB: 3D tracking of hands and hand-object interactions based on short-baseline stereo
• Two-hand Global 3D Pose Estimation Using Monocular RGB
• SeqHAND:RGB-Sequence-Based 3D Hand Pose and Shape Estimation
• Temporal-Aware Self-Supervised Learning for 3D Hand Pose and Mesh Estimation in Videos
• Two-Stream Binocular Network: Accurate Near Field Finger Detection Based On Binocular Images
• MVHM: A Large-Scale Multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation
• Fast Monocular Hand Pose Estimation on Embedded Systems
• 3D Hand Pose Estimation via Regularized Graph Representation Learning
• Synthetic Training for Monocular Human Mesh Recovery
• Parallel mesh reconstruction streams for pose estimation of interacting hands
• Explicit Spatiotemporal Joint Relation Learning for Tracking Human Pose
• DGGAN: Depth-image Guided Generative Adversarial Networks for Disentangling RGB and Depth Images in 3D Hand Pose Estimation
• Normalized Diversification