Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer

Abstract

Robot perception systems need to perform reliable image segmentation in real-time on noisy, raw perception data. State-of-the-art segmentation approaches use large CNN models and carefully constructed datasets; however, these models focus on accuracy at the cost of real-time inference. Furthermore, the standard semantic segmentation datasets are not large enough for training CNNs without augmentation and are not representative of noisy, uncurated robot perception data. We propose improving the performance of real-time segmentation frameworks on robot perception data by transferring features learned from synthetic segmentation data. We show that pretraining real-time segmentation architectures with synthetic segmentation data instead of ImageNet improves fine-tuning performance by reducing the bias learned in pretraining and closing the \textit{transfer gap} as a result. Our experiments show that our real-time robot perception models pretrained on synthetic data outperform those pretrained on ImageNet for every scale of fine-tuning data examined. Moreover, the degree to which synthetic pretraining outperforms ImageNet pretraining increases as the availability of robot data decreases, making our approach attractive for robotics domains where dataset collection is hard and/or expensive.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Review on Deep Learning Techniques Applied to Semantic Segmentation
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• An Analysis of Deep Neural Network Models for Practical Applications
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• Playing for Data: Ground Truth from Computer Games
• Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
• SceneNet RGB-D: 5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth
• SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks

Cited by in corpus (1)

• Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection