Analysis of the hands in egocentric vision: A survey

Abstract

Egocentric vision (a.k.a. first-person vision - FPV) applications have thrived over the past few years, thanks to the availability of affordable wearable cameras and large annotated datasets. The position of the wearable camera (usually mounted on the head) allows recording exactly what the camera wearers have in front of them, in particular hands and manipulated objects. This intrinsic advantage enables the study of the hands from multiple perspectives: localizing hands and their parts within the images; understanding what actions and activities the hands are involved in; and developing human-computer interfaces that rely on hand gestures. In this survey, we review the literature that focuses on the hands using egocentric vision, categorizing the existing approaches into: localization (where are the hands or parts of them?); interpretation (what are the hands doing?); and application (e.g., systems that used egocentric hand cues for solving a specific problem). Moreover, a list of the most prominent datasets with hand-based annotations is provided.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Fully Convolutional Networks for Semantic Segmentation
• Determinantal point processes for machine learning
• The Evolution of First Person Vision Methods: A Survey
• Left/Right Hand Segmentation in Egocentric Videos
• Beyond One Glance: Gated Recurrent Architecture for Hand Segmentation

Cited by in corpus (7)

• Unified Learning Approach for Egocentric Hand Gesture Recognition and Fingertip Detection
• Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation
• In the Eye of the Beholder: Gaze and Actions in First Person Video
• Recognizing Hand Use and Hand Role at Home After Stroke from Egocentric Video
• Testing Human-Hand Segmentation on In-Distribution and Out-of-Distribution Data in Human-Robot Interactions Using a Deep Ensemble Model
• Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database with Automatic Labelling
• Measuring hand use in the home after cervical spinal cord injury using egocentric video