Learning Deep Object Detectors from 3D Models

Abstract

Crowdsourced 3D CAD models are becoming easily accessible online, and can potentially generate an infinite number of training images for almost any object category.We show that augmenting the training data of contemporary Deep Convolutional Neural Net (DCNN) models with such synthetic data can be effective, especially when real training data is limited or not well matched to the target domain. Most freely available CAD models capture 3D shape but are often missing other low level cues, such as realistic object texture, pose, or background. In a detailed analysis, we use synthetic CAD-rendered images to probe the ability of DCNN to learn without these cues, with surprising findings. In particular, we show that when the DCNN is fine-tuned on the target detection task, it exhibits a large degree of invariance to missing low-level cues, but, when pretrained on generic ImageNet classification, it learns better when the low-level cues are simulated. We show that our synthetic DCNN training approach significantly outperforms previous methods on the PASCAL VOC2007 dataset when learning in the few-shot scenario and improves performance in a domain shift scenario on the Office benchmark.

References in corpus (2)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition

Cited by in corpus (13)

• Learning from Synthetic Humans
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
• Learning Dense Correspondence via 3D-guided Cycle Consistency
• DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
• Deep Learning for Single-View Instance Recognition
• Metamorphic Testing for Object Detection Systems
• Synthetic to Real Adaptation with Generative Correlation Alignment Networks
• Recovering Homography from Camera Captured Documents using Convolutional Neural Networks
• Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery
• Context Augmentation for Convolutional Neural Networks
• What can we learn about CNNs from a large scale controlled object dataset?
• See the Difference: Direct Pre-Image Reconstruction and Pose Estimation by Differentiating HOG