On Rendering Synthetic Images for Training an Object Detector

Abstract

We propose a novel approach to synthesizing images that are effective for training object detectors. Starting from a small set of real images, our algorithm estimates the rendering parameters required to synthesize similar images given a coarse 3D model of the target object. These parameters can then be reused to generate an unlimited number of training images of the object of interest in arbitrary 3D poses, which can then be used to increase classification performances. A key insight of our approach is that the synthetically generated images should be similar to real images, not in terms of image quality, but rather in terms of features used during the detector training. We show in the context of drone, plane, and car detection that using such synthetically generated images yields significantly better performances than simply perturbing real images or even synthesizing images in such way that they look very realistic, as is often done when only limited amounts of training data are available.

Cited by in corpus (14)

• Beyond Sharing Weights for Deep Domain Adaptation
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Generating Synthetic Data for Text Recognition
• Metamorphic Testing for Object Detection Systems
• Sim-to-Real Learning for Casualty Detection from Ground Projected Point Cloud Data
• Cut-and-Paste Dataset Generation for Balancing Domain Gaps in Object Instance Detection
• Frankenstein: Learning Deep Face Representations using Small Data
• Synthetic Data Generation and Adaption for Object Detection in Smart Vending Machines
• HWNet v2: An Efficient Word Image Representation for Handwritten Documents
• DiffRenderGAN: Addressing Training Data Scarcity in Deep Segmentation Networks for Quantitative Nanomaterial Analysis through Differentiable Rendering and Generative Modelling
• Utilizing synthetic training data for the supervised classification of rat ultrasonic vocalizations
• Single Image Reflection Removal with Physically-Based Training Images
• Identity Enhanced Residual Image Denoising
• From Virtual to Real World Visual Perception using Domain Adaptation -- The DPM as Example