Enhanced Touchable Projector-depth System with Deep Hand Pose Estimation
arXiv:1812.11090
Abstract
Touchable projection with structured light range cameras is a prolific medium for large interaction surfaces, affording multiple simultaneous users and simple, cheap setup. However robust touch detection in such projector-depth systems is difficult to achieve due to measurement noise. We propose a novel combination of surface touch detection and a deep network for hand pose estimation, which aids in detecting both on- and above-surface hand gestures, disambiguating multiple touch fingers, as well as recovering fingertip positions in face of noisy input. We present the details of our GPU-accelerated system and an evaluation of its performance, as well as applications such as an enhanced virtual keyboard that utilizes the added features.
arXiv comment9 pages, 15 figures
References in corpus (5)
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Hands Deep in Deep Learning for Hand Pose Estimation
- Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNs
- 3D Hand Pose Detection in Egocentric RGB-D Images
- Depth-based hand pose estimation: methods, data, and challenges