Domain-Scalable Unpaired Image Translation via Latent Space Anchoring

Abstract

Unpaired image-to-image translation (UNIT) aims to map images between two visual domains without paired training data. However, given a UNIT model trained on certain domains, it is difficult for current methods to incorporate new domains because they often need to train the full model on both existing and new domains. To address this problem, we propose a new domain-scalable UNIT method, termed as latent space anchoring, which can be efficiently extended to new visual domains and does not need to fine-tune encoders and decoders of existing domains. Our method anchors images of different domains to the same latent space of frozen GANs by learning lightweight encoder and regressor models to reconstruct single-domain images. In the inference phase, the learned encoders and decoders of different domains can be arbitrarily combined to translate images between any two domains without fine-tuning. Experiments on various datasets show that the proposed method achieves superior performance on both standard and domain-scalable UNIT tasks in comparison with the state-of-the-art methods.

arXiv commentAccepeted to IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Code is available at https://github.com/siyuhuang/Latent-Space-Anchoring

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Alias-Free Generative Adversarial Networks
• U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
• Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs
• One-Shot Unsupervised Cross Domain Translation
• Closed-Form Factorization of Latent Semantics in GANs
• Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation
• GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modeling
• SDIT: Scalable and Diverse Cross-domain Image Translation