BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis

Abstract

In this paper we introduce a large-scale hand pose dataset, collected using a novel capture method. Existing datasets are either generated synthetically or captured using depth sensors: synthetic datasets exhibit a certain level of appearance difference from real depth images, and real datasets are limited in quantity and coverage, mainly due to the difficulty to annotate them. We propose a tracking system with six 6D magnetic sensors and inverse kinematics to automatically obtain 21-joints hand pose annotations of depth maps captured with minimal restriction on the range of motion. The capture protocol aims to fully cover the natural hand pose space. As shown in embedding plots, the new dataset exhibits a significantly wider and denser range of hand poses compared to existing benchmarks. Current state-of-the-art methods are evaluated on the dataset, and we demonstrate significant improvements in cross-benchmark performance. We also show significant improvements in egocentric hand pose estimation with a CNN trained on the new dataset.

References in corpus (2)

• Hands Deep in Deep Learning for Hand Pose Estimation
• Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNs

Cited by in corpus (14)

• V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map
• First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations
• Model-based Hand Pose Estimation for Generalized Hand Shape with Appearance Normalization
• Floor-SP: Inverse CAD for Floorplans by Sequential Room-wise Shortest Path
• Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation
• DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth
• Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
• Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layer
• Dense 3D Regression for Hand Pose Estimation
• MVHM: A Large-Scale Multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation
• Dual Grid Net: hand mesh vertex regression from single depth maps
• Egocentric 6-DoF Tracking of Small Handheld Objects
• Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation
• Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images