BusyHands: A Hand-Tool Interaction Database for Assembly Tasks Semantic Segmentation

Abstract

Visual segmentation has seen tremendous advancement recently with ready solutions for a wide variety of scene types, including human hands and other body parts. However, focus on segmentation of human hands while performing complex tasks, such as manual assembly, is still severely lacking. Segmenting hands from tools, work pieces, background and other body parts is extremely difficult because of self-occlusions and intricate hand grips and poses. In this paper we introduce BusyHands, a large open dataset of pixel-level annotated images of hands performing 13 different tool-based assembly tasks, from both real-world captures and virtual-world renderings. A total of 7906 samples are included in our first-in-kind dataset, with both RGB and depth images as obtained from a Kinect V2 camera and Blender. We evaluate several state-of-the-art semantic segmentation methods on our dataset as a proposed performance benchmark.

10 pages, 8 figures

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Deep Residual Learning for Image Recognition
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Pyramid Scene Parsing Network
• Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes