Mask Based Unsupervised Content Transfer
arXiv:1906.06558
Abstract
We consider the problem of translating, in an unsupervised manner, between two domains where one contains some additional information compared to the other. The proposed method disentangles the common and separate parts of these domains and, through the generation of a mask, focuses the attention of the underlying network to the desired augmentation alone, without wastefully reconstructing the entire target. This enables state-of-the-art quality and variety of content translation, as demonstrated through extensive quantitative and qualitative evaluation. Our method is also capable of adding the separate content of different guide images and domains as well as remove existing separate content. Furthermore, our method enables weakly-supervised semantic segmentation of the separate part of each domain, where only class labels are provided. Our code is publicly available at https://github.com/rmokady/mbu-content-tansfer.
References in corpus (11)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Toward Multimodal Image-to-Image Translation
- Unsupervised Attention-guided Image to Image Translation
- Demystifying MMD GANs
- Neural Style Transfer: A Review
- Audio style transfer
- Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation
- Weakly Supervised Instance Segmentation using Class Peak Response
- Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency
- Emerging Disentanglement in Auto-Encoder Based Unsupervised Image Content Transfer
- Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation