Efficient Realistic Data Generation Framework leveraging Deep Learning-based Human Digitization

Abstract

The performance of supervised deep learning algorithms depends significantly on the scale, quality and diversity of the data used for their training. Collecting and manually annotating large amount of data can be both time-consuming and costly tasks to perform. In the case of tasks related to visual human-centric perception, the collection and distribution of such data may also face restrictions due to legislation regarding privacy. In addition, the design and testing of complex systems, e.g., robots, which often employ deep learning-based perception models, may face severe difficulties as even state-of-the-art methods trained on real and large-scale datasets cannot always perform adequately due to not having been adapted to the visual differences between the virtual and the real world data. As an attempt to tackle and mitigate the effect of these issues, we present a method that automatically generates realistic synthetic data with annotations for a) person detection, b) face recognition, and c) human pose estimation. The proposed method takes as input real background images and populates them with human figures in various poses. Instead of using hand-made 3D human models, we propose the use of models generated through deep learning methods, further reducing the dataset creation costs, while maintaining a high level of realism. In addition, we provide open-source and easy to use tools that implement the proposed pipeline, allowing for generating highly-realistic synthetic datasets for a variety of tasks. A benchmarking and evaluation in the corresponding tasks shows that synthetic data can be effectively used as a supplement to real data.

References in corpus (12)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Deep Residual Learning for Image Recognition
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• CosFace: Large Margin Cosine Loss for Deep Face Recognition
• MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
• Invertible Residual Networks
• PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
• Learn to Combine Modalities in Multimodal Deep Learning
• MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices
• Distribution-Aware Coordinate Representation for Human Pose Estimation
• An Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance Detection
• How useful is photo-realistic rendering for visual learning?

Cited by in corpus (1)

• Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback Datasets