Training a Feedback Loop for Hand Pose Estimation

Abstract

We propose an entirely data-driven approach to estimating the 3D pose of a hand given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. They remove the need for fitting a 3D model to the input data, which requires both a carefully designed fitting function and algorithm. We show that our approach outperforms state-of-the-art methods, and is efficient as our implementation runs at over 400 fps on a single GPU.

Presented at ICCV 2015 (oral)

References in corpus (4)

• Hands Deep in Deep Learning for Hand Pose Estimation
• Learning Human Pose Estimation Features with Convolutional Networks
• Learning Generative Models with Visual Attention
• Capturing Hand Motion with an RGB-D Sensor, Fusing a Generative Model with Salient Points

Cited by in corpus (8)

• Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
• Geometric Back-projection Network for Point Cloud Classification
• PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
• Simultaneous Hand Pose and Skeleton Bone-Lengths Estimation from a Single Depth Image
• DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth
• Back to RGB: 3D tracking of hands and hand-object interactions based on short-baseline stereo
• Dense 3D Regression for Hand Pose Estimation
• Augmented Reality Oculus Rift