Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layer

Abstract

Recently, 3D input data based hand pose estimation methods have shown state-of-the-art performance, because 3D data capture more spatial information than the depth image. Whereas 3D voxel-based methods need a large amount of memory, PointNet based methods need tedious preprocessing steps such as K-nearest neighbour search for each point. In this paper, we present a novel deep learning hand pose estimation method for an unordered point cloud. Our method takes 1024 3D points as input and does not require additional information. We use Permutation Equivariant Layer (PEL) as the basic element, where a residual network version of PEL is proposed for the hand pose estimation task. Furthermore, we propose a voting based scheme to merge information from individual points to the final pose output. In addition to the pose estimation task, the voting-based scheme can also provide point cloud segmentation result without ground-truth for segmentation. We evaluate our method on both NYU dataset and the Hands2017Challenge dataset. Our method outperforms recent state-of-the-art methods, where our pose accuracy is currently the best for the Hands2017Challenge dataset.

References in corpus (12)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Embodied Hands: Modeling and Capturing Hands and Bodies Together
• Hands Deep in Deep Learning for Hand Pose Estimation
• Dynamic Graph CNN for Learning on Point Clouds
• Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation
• SO-Net: Self-Organizing Network for Point Cloud Analysis
• Hand3D: Hand Pose Estimation using 3D Neural Network
• Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNs
• The 2017 Hands in the Million Challenge on 3D Hand Pose Estimation
• Triplet-Center Loss for Multi-View 3D Object Retrieval
• V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map
• BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis

Cited by in corpus (2)

• Temporal-Aware Self-Supervised Learning for 3D Hand Pose and Mesh Estimation in Videos
• DGGAN: Depth-image Guided Generative Adversarial Networks for Disentangling RGB and Depth Images in 3D Hand Pose Estimation