Dense 3D Regression for Hand Pose Estimation

Abstract

We present a simple and effective method for 3D hand pose estimation from a single depth frame. As opposed to previous state-of-the-art methods based on holistic 3D regression, our method works on dense pixel-wise estimation. This is achieved by careful design choices in pose parameterization, which leverages both 2D and 3D properties of depth map. Specifically, we decompose the pose parameters into a set of per-pixel estimations, i.e., 2D heat maps, 3D heat maps and unit 3D directional vector fields. The 2D/3D joint heat maps and 3D joint offsets are estimated via multi-task network cascades, which is trained end-to-end. The pixel-wise estimations can be directly translated into a vote casting scheme. A variant of mean shift is then used to aggregate local votes while enforcing consensus between the the estimated 3D pose and the pixel-wise 2D and 3D estimations by design. Our method is efficient and highly accurate. On MSRA and NYU hand dataset, our method outperforms all previous state-of-the-art approaches by a large margin. On the ICVL hand dataset, our method achieves similar accuracy compared to the currently proposed nearly saturated result and outperforms various other proposed methods. Code is available $\text{href}$.

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
• Stacked Hourglass Networks for Human Pose Estimation
• Hands Deep in Deep Learning for Hand Pose Estimation
• Convolutional Pose Machines
• Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
• Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation
• A simple yet effective baseline for 3d human pose estimation
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• Compositional Human Pose Regression
• BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis
• Holistic Planimetric prediction to Local Volumetric prediction for 3D Human Pose Estimation
• Training a Feedback Loop for Hand Pose Estimation
• 3D Human Pose Estimation from a Single Image via Distance Matrix Regression

Cited by in corpus (1)

• DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth