Integrating Categorical Semantics into Unsupervised Domain Translation

Abstract

While unsupervised domain translation (UDT) has seen a lot of success recently, we argue that mediating its translation via categorical semantic features could broaden its applicability. In particular, we demonstrate that categorical semantics improves the translation between perceptually different domains sharing multiple object categories. We propose a method to learn, in an unsupervised manner, categorical semantic features (such as object labels) that are invariant of the source and target domains. We show that conditioning the style encoder of unsupervised domain translation methods on the learned categorical semantics leads to a translation preserving the digits on MNIST$\leftrightarrow$SVHN and to a more realistic stylization on Sketches$\to$Reals.

arXiv comment22 pages. In submission to the International Conference on Learning Representation (ICLR) 2021

References in corpus (17)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Improved Baselines with Momentum Contrastive Learning
• Learning deep representations by mutual information estimation and maximization
• Toward Multimodal Image-to-Image Translation
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• Learning Discrete Representations via Information Maximizing Self-Augmented Training
• InstaGAN: Instance-aware Image-to-Image Translation
• StarGAN v2: Diverse Image Synthesis for Multiple Domains
• Virtual Mixup Training for Unsupervised Domain Adaptation
• Emerging Disentanglement in Auto-Encoder Based Unsupervised Image Content Transfer
• Optimal Unsupervised Domain Translation
• The Role of Minimal Complexity Functions in Unsupervised Learning of Semantic Mappings
• Semantics-Aware Image to Image Translation and Domain Transfer