Few-Shot Unsupervised Image-to-Image Translation

Abstract

Unsupervised image-to-image translation methods learn to map images in a given class to an analogous image in a different class, drawing on unstructured (non-registered) datasets of images. While remarkably successful, current methods require access to many images in both source and destination classes at training time. We argue this greatly limits their use. Drawing inspiration from the human capability of picking up the essence of a novel object from a small number of examples and generalizing from there, we seek a few-shot, unsupervised image-to-image translation algorithm that works on previously unseen target classes that are specified, at test time, only by a few example images. Our model achieves this few-shot generation capability by coupling an adversarial training scheme with a novel network design. Through extensive experimental validation and comparisons to several baseline methods on benchmark datasets, we verify the effectiveness of the proposed framework. Our implementation and datasets are available at https://github.com/NVlabs/FUNIT .

arXiv commentThe paper will be presented at the International Conference on Computer Vision (ICCV) 2019

References in corpus (13)

• Prototypical Networks for Few-shot Learning
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Toward Multimodal Image-to-Image Translation
• Which Training Methods for GANs do actually Converge?
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• On First-Order Meta-Learning Algorithms
• Revisiting Batch Normalization For Practical Domain Adaptation
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• One-Shot Unsupervised Cross Domain Translation
• Unsupervised Multi-Domain Image Translation with Domain-Specific Encoders/Decoders

Cited by in corpus (29)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Unpaired Motion Style Transfer from Video to Animation
• Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning
• MelGAN-VC: Voice Conversion and Audio Style Transfer on arbitrarily long samples using Spectrograms
• One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
• Automated Relational Meta-learning
• MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targets
• Dynamic Instance Normalization for Arbitrary Style Transfer
• Towards Realistic 3D Embedding via View Alignment
• Only a Matter of Style: Age Transformation Using a Style-Based Regression Model
• The Six Fronts of the Generative Adversarial Networks
• Few-shot Image Generation via Cross-domain Correspondence
• Variational Disentanglement for Domain Generalization
• Few-shot Video-to-Video Synthesis
• Adversarial Image Composition with Auxiliary Illumination
• GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images
• Deep Line Art Video Colorization with a Few References
• Transflow Learning: Repurposing Flow Models Without Retraining
• F2GAN: Fusing-and-Filling GAN for Few-shot Image Generation
• GANMEX: One-vs-One Attributions Guided by GAN-based Counterfactual Explanation Baselines
• Domain-Specific Mappings for Generative Adversarial Style Transfer
• Augmentation-Interpolative AutoEncoders for Unsupervised Few-Shot Image Generation
• Cross-Spectral Face Hallucination via Disentangling Independent Factors
• One-Shot Domain Adaptation For Face Generation
• Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation
• Few-Shot Unsupervised Image-to-Image Translation on complex scenes
• Brain Image Synthesis with Unsupervised Multivariate Canonical CSC${\ell }_4$Net
• Delving into Rectifiers in Style-Based Image Translation
• Cross-View Exocentric to Egocentric Video Synthesis