Synthetic to Real Adaptation with Generative Correlation Alignment Networks

Abstract

Synthetic images rendered from 3D CAD models are useful for augmenting training data for object recognition algorithms. However, the generated images are non-photorealistic and do not match real image statistics. This leads to a large domain discrepancy, causing models trained on synthetic data to perform poorly on real domains. Recent work has shown the great potential of deep convolutional neural networks to generate realistic images, but has not utilized generative models to address synthetic-to-real domain adaptation. In this work, we propose a Deep Generative Correlation Alignment Network (DGCAN) to synthesize images using a novel domain adaption algorithm. DGCAN leverages a shape preserving loss and a low level statistic matching loss to minimize the domain discrepancy between synthetic and real images in deep feature space. Experimentally, we show training off-the-shelf classifiers on the newly generated data can significantly boost performance when testing on the real image domains (PASCAL VOC 2007 benchmark and Office dataset), improving upon several existing methods.

13 pages, 8 figures

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Conditional Generative Adversarial Nets
• Deep Domain Confusion: Maximizing for Domain Invariance
• Equivalence of distance-based and RKHS-based statistics in hypothesis testing
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• Deep CORAL: Correlation Alignment for Deep Domain Adaptation
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Playing for Data: Ground Truth from Computer Games
• Preserving Color in Neural Artistic Style Transfer
• Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
• Return of Frustratingly Easy Domain Adaptation
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• Learning Deep Object Detectors from 3D Models

Cited by in corpus (5)

• Deep Visual Domain Adaptation: A Survey
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Meta Networks for Neural Style Transfer
• GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB
• Unsupervised Domain Adaptation using Generative Models and Self-ensembling