Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery
arXiv:1711.09082
Abstract
In human learning, it is common to use multiple sources of information jointly. However, most existing feature learning approaches learn from only a single task. In this paper, we propose a novel multi-task deep network to learn generalizable high-level visual representations. Since multi-task learning requires annotations for multiple properties of the same training instance, we look to synthetic images to train our network. To overcome the domain difference between real and synthetic data, we employ an unsupervised feature space domain adaptation method based on adversarial learning. Given an input synthetic RGB image, our network simultaneously predicts its surface normal, depth, and instance contour, while also minimizing the feature space domain differences between real and synthetic data. Through extensive experiments, we demonstrate that our network learns more transferable representations compared to single-task baselines. Our learned representation produces state-of-the-art transfer learning results on PASCAL VOC 2007 classification and 2012 detection.
References in corpus (22)
- Unsupervised Domain Adaptation by Backpropagation
- Multi-Scale Context Aggregation by Dilated Convolutions
- Fully Convolutional Networks for Semantic Segmentation
- Adversarial Feature Learning
- Adversarial Discriminative Domain Adaptation
- Virtual Worlds as Proxy for Multi-Object Tracking Analysis
- Unsupervised Visual Representation Learning by Context Prediction
- Unsupervised Learning by Predicting Noise
- Unsupervised Learning of Visual Representations using Videos
- Data-dependent Initializations of Convolutional Neural Networks
- Places: An Image Database for Deep Scene Understanding
- Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
- Playing for Data: Ground Truth from Computer Games
- Learning to See by Moving
- R-CNNs for Pose Estimation and Action Detection
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
- PixelNet: Representation of the pixels, by the pixels, and for the pixels
- Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
- Semantic Scene Completion from a Single Depth Image
- Learning Deep Object Detectors from 3D Models
- Learning Features by Watching Objects Move
- Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction