Training Generative Adversarial Networks from Incomplete Observations using Factorised Discriminators
arXiv:1905.12660
Abstract
Generative adversarial networks (GANs) have shown great success in applications such as image generation and inpainting. However, they typically require large datasets, which are often not available, especially in the context of prediction tasks such as image segmentation that require labels. Therefore, methods such as the CycleGAN use more easily available unlabelled data, but do not offer a way to leverage additional labelled data for improved performance. To address this shortcoming, we show how to factorise the joint data distribution into a set of lower-dimensional distributions along with their dependencies. This allows splitting the discriminator in a GAN into multiple "sub-discriminators" that can be independently trained from incomplete observations. Their outputs can be combined to estimate the density ratio between the joint real and the generator distribution, which enables training generators as in the original GAN framework. We apply our method to image generation, image segmentation and audio source separation, and obtain improved performance over a standard GAN when additional incomplete training examples are available. For the Cityscapes segmentation task in particular, our method also improves accuracy by an absolute 14.9% over CycleGAN while using only 25 additional paired examples.
arXiv comment10 pages plus 14 pages appendix. Accepted at the International Conference on Learning Representations (ICLR) 2020. Camera-ready submission. Implementation available at https://github.com/f90/FactorGAN
References in corpus (9)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Spectral Normalization for Generative Adversarial Networks
- GAIN: Missing Data Imputation using Generative Adversarial Nets
- Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
- Amortised MAP Inference for Image Super-resolution
- Learning Independent Features with Adversarial Nets for Non-linear ICA
- JointGAN: Multi-Domain Joint Distribution Learning with Generative Adversarial Nets
- Adversarial Message Passing For Graphical Models
- Weakly Supervised Audio Source Separation via Spectrum Energy Preserved Wasserstein Learning