Fast and Robust Hand Tracking Using Detection-Guided Optimization
arXiv:1602.04124 · doi:10.1109/CVPR.2015.7298941
Abstract
Markerless tracking of hands and fingers is a promising enabler for human-computer interaction. However, adoption has been limited because of tracking inaccuracies, incomplete coverage of motions, low framerate, complex camera setups, and high computational requirements. In this paper, we present a fast method for accurately tracking rapid and complex articulations of the hand using a single depth camera. Our algorithm uses a novel detection-guided optimization strategy that increases the robustness and speed of pose estimation. In the detection step, a randomized decision forest classifies pixels into parts of the hand. In the optimization step, a novel objective function combines the detected part labels and a Gaussian mixture representation of the depth to estimate a pose that best fits the depth. Our approach needs comparably less computational resources which makes it extremely fast (50 fps without GPU support). The approach also supports varying static, or moving, camera-to-scene arrangements. We show the benefits of our method by evaluating on public datasets and comparing against previous work.
9 pages, Accepted version of paper published at CVPR 2015
References in corpus (2)
Cited by in corpus (35)
- Embodied Hands: Modeling and Capturing Hands and Bodies Together
- Capturing Hands in Action using Discriminative Salient Points and Physics Simulation
- Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
- Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation
- Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera
- 3D Hand Pose Tracking and Estimation Using Stereo Matching
- FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integration
- Personal Privacy Protection via Irrelevant Faces Tracking and Pixelation in Video Live Streaming
- ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild
- RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video
- Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input
- Monocular Total Capture: Posing Face, Body, and Hands in the Wild
- An FCNN-Based Super-Resolution mmWave Radar Framework for Contactless Musical Instrument Interface
- Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera
- Using CNNs For Users Segmentation In Video See-Through Augmented Virtuality
- End-to-end Hand Mesh Recovery from a Monocular RGB Image
- Single-Network Whole-Body Pose Estimation
- Ego2Hands: A Dataset for Egocentric Two-hand Segmentation and Detection
- Direction matters: hand pose estimation from local surface normals
- GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB
- Simultaneous Hand Pose and Skeleton Bone-Lengths Estimation from a Single Depth Image
- Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data
- Model-based Hand Pose Estimation for Generalized Hand Shape with Appearance Normalization
- Model-based Outdoor Performance Capture
- EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras
- Real-Time Hand Gesture Identification in Thermal Images
- EasyHOI: Unleashing the Power of Large Models for Reconstructing Hand-Object Interactions in the Wild
- Back to RGB: 3D tracking of hands and hand-object interactions based on short-baseline stereo
- Whole-Body Human Pose Estimation in the Wild
- HMTNet:3D Hand Pose Estimation from Single Depth Image Based on Hand Morphological Topology
- Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation
- Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
- Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation
- A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation
- EgoRenderer: Rendering Human Avatars from Egocentric Camera Images