Unity Perception: Generate Synthetic Data for Computer Vision

Abstract

We introduce the Unity Perception package which aims to simplify and accelerate the process of generating synthetic datasets for computer vision tasks by offering an easy-to-use and highly customizable toolset. This open-source package extends the Unity Editor and engine components to generate perfectly annotated examples for several common computer vision tasks. Additionally, it offers an extensible Randomization framework that lets the user quickly construct and configure randomized simulation parameters in order to introduce variation into the generated datasets. We provide an overview of the provided tools and how they work, and demonstrate the value of the generated synthetic datasets by training a 2D object detection model. The model trained with mostly synthetic data outperforms the model trained using only real data.

arXiv commentWe corrected tasks supported by NVISII platform. For the Unity perception package, see https://github.com/Unity-Technologies/com.unity.perception

References in corpus (7)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Rethinking Atrous Convolution for Semantic Image Segmentation
• The VIA Annotation Software for Images, Audio and Video
• Solving Rubik's Cube with a Robot Hand
• AI2-THOR: An Interactive 3D Environment for Visual AI
• iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes
• NViSII: A Scriptable Tool for Photorealistic Image Generation

Cited by in corpus (5)

• Technology Readiness Levels for Machine Learning Systems
• A survey of synthetic data augmentation methods in computer vision
• Methodology for generating synthetic labeled datasets for visual container inspection
• Supervised Representation Learning towards Generalizable Assembly State Recognition
• MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis