UnrealStereo: Controlling Hazardous Factors to Analyze Stereo Vision

Abstract

A reliable stereo algorithm is critical for many robotics applications. But textureless and specular regions can easily cause failure by making feature matching difficult. Understanding whether an algorithm is robust to these hazardous regions is important. Although many stereo benchmarks have been developed to evaluate performance, it is hard to quantify the effect of hazardous regions in real images because the location and severity of these regions are unknown. In this paper, we develop a synthetic image generation tool enabling to control hazardous factors, such as making objects more specular or transparent, to produce hazardous regions at different degrees. The densely controlled sampling strategy in virtual worlds enables to effectively stress test stereo algorithms by varying the types and degrees of the hazard. We generate a large synthetic image dataset with automatically computed hazardous regions and analyze algorithms on these regions. The observations from synthetic images are further validated by annotating hazardous regions in real-world datasets Middlebury and KITTI (which gives a sparse sampling of the hazards). Our synthetic image generation tool is based on a game engine Unreal Engine 4 and will be open-source along with the virtual scenes in our experiments. Many publicly available realistic game contents can be used by our tool to provide an enormous resource for development and evaluation of algorithms.

3DV 2018 (oral)

References in corpus (6)

• Computing the Stereo Matching Cost with a Convolutional Neural Network
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Playing for Data: Ground Truth from Computer Games
• Adversarial Examples for Semantic Segmentation and Object Detection
• Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications
• UnrealCV: Connecting Computer Vision to Unreal Engine

Cited by in corpus (5)

• InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset
• Adversarial Attacks Beyond the Image Space
• SampleAhead: Online Classifier-Sampler Communication for Learning from Synthesized Data
• A study of the effect of the illumination model on the generation of synthetic training datasets
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation