A survey of synthetic data augmentation methods in computer vision

Abstract

The standard approach to tackling computer vision problems is to train deep convolutional neural network (CNN) models using large-scale image datasets which are representative of the target task. However, in many scenarios, it is often challenging to obtain sufficient image data for the target task. Data augmentation is a way to mitigate this challenge. A common practice is to explicitly transform existing images in desired ways so as to create the required volume and variability of training data necessary to achieve good generalization performance. In situations where data for the target domain is not accessible, a viable workaround is to synthesize training data from scratch--i.e., synthetic data augmentation. This paper presents an extensive review of synthetic data augmentation techniques. It covers data synthesis approaches based on realistic 3D graphics modeling, neural style transfer (NST), differential neural rendering, and generative artificial intelligence (AI) techniques such as generative adversarial networks (GANs) and variational autoencoders (VAEs). For each of these classes of methods, we focus on the important data generation and augmentation techniques, general scope of application and specific use-cases, as well as existing limitations and possible workarounds. Additionally, we provide a summary of common synthetic datasets for training computer vision models, highlighting the main features, application domains and supported tasks. Finally, we discuss the effectiveness of synthetic data augmentation methods. Since this is the first paper to explore synthetic data augmentation methods in great detail, we are hoping to equip readers with the necessary background information and in-depth knowledge of existing methods and their attendant issues.

References in corpus (29)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Implicit Neural Representations with Periodic Activation Functions
• Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
• MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
• FusionNet: 3D Object Classification Using Multiple Data Representations
• Laplacian-Steered Neural Style Transfer
• Occlusion Handling in Generic Object Detection: A Review
• Unity Perception: Generate Synthetic Data for Computer Vision
• Uncertainty Modeling for Out-of-Distribution Generalization
• Consistency Regularization for Generative Adversarial Networks
• Object Detection Using Deep CNNs Trained on Synthetic Images
• DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering
• Lagrangian Neural Style Transfer for Fluids
• Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments
• On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
• NeRF-VAE: A Geometry Aware 3D Scene Generative Model
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• HandAugment: A Simple Data Augmentation Method for Depth-Based 3D Hand Pose Estimation
• SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo
• ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations
• Data Augmentation for Object Detection via Differentiable Neural Rendering
• Automatic Generation of Machine Learning Synthetic Data Using ROS
• Neural Photometry-guided Visual Attribute Transfer
• Random Shadows and Highlights: A new data augmentation method for extreme lighting conditions
• Hallucinated Adversarial Learning for Robust Visual Tracking
• Controllable Data Augmentation Through Deep Relighting
• Cross-Domain Image Classification through Neural-Style Transfer Data Augmentation