T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks
arXiv:1808.01454
Abstract
Current methods for single-image depth estimation use training datasets with real image-depth pairs or stereo pairs, which are not easy to acquire. We propose a framework, trained on synthetic image-depth pairs and unpaired real images, that comprises an image translation network for enhancing realism of input images, followed by a depth prediction network. A key idea is having the first network act as a wide-spectrum input translator, taking in either synthetic or real images, and ideally producing minimally modified realistic images. This is done via a reconstruction loss when the training input is real, and GAN loss when synthetic, removing the need for heuristic self-regularization. The second network is trained on a task loss for synthetic image-depth pairs, with extra GAN loss to unify real and synthetic feature distributions. Importantly, the framework can be trained end-to-end, leading to good results, even surpassing early deep-learning methods that use real paired data.
arXiv comment15 pages, 8 figures
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Unsupervised Image-to-Image Translation Networks
- DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
- Virtual Worlds as Proxy for Multi-Object Tracking Analysis
- Depth Extraction from Video Using Non-parametric Sampling
- Variational Approaches for Auto-Encoding Generative Adversarial Networks
- Semi-Supervised Deep Learning for Monocular Depth Map Prediction