Structured Generative Adversarial Networks

Abstract

We study the problem of conditional generative modeling based on designated semantics or structures. Existing models that build conditional generators either require massive labeled instances as supervision or are unable to accurately control the semantics of generated samples. We propose structured generative adversarial networks (SGANs) for semi-supervised conditional generative modeling. SGAN assumes the data x is generated conditioned on two independent latent variables: y that encodes the designated semantics, and z that contains other factors of variation. To ensure disentangled semantics in y and z, SGAN builds two collaborative games in the hidden space to minimize the reconstruction error of y and z, respectively. Training SGAN also involves solving two adversarial games that have their equilibrium concentrating at the true joint data distributions p(x, z) and p(x, y), avoiding distributing the probability mass diffusely over data space that MLE-based methods may suffer. We assess SGAN by evaluating its trained networks, and its performance on downstream tasks. We show that SGAN delivers a highly controllable generator, and disentangled representations; it also establishes start-of-the-art results across multiple datasets when applied for semi-supervised image classification (1.27%, 5.73%, 17.26% error rates on MNIST, SVHN and CIFAR-10 using 50, 1000 and 4000 labels, respectively). Benefiting from the separate modeling of y and z, SGAN can generate images with high visual quality and strictly following the designated semantic, and can be extended to a wide spectrum of applications, such as style transfer.

To appear in NIPS 2017

References in corpus (14)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Temporal Ensembling for Semi-Supervised Learning
• Improved Training of Wasserstein GANs
• Improved Techniques for Training GANs
• Adversarial Feature Learning
• Adversarially Learned Inference
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Auxiliary Deep Generative Models
• Triple Generative Adversarial Nets
• Toward Controlled Generation of Text
• Recurrent Topic-Transition GAN for Visual Paragraph Generation
• Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines
• ZM-Net: Real-time Zero-shot Image Manipulation Network

Cited by in corpus (3)

• Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion
• NIPS - Not Even Wrong? A Systematic Review of Empirically Complete Demonstrations of Algorithmic Effectiveness in the Machine Learning and Artificial Intelligence Literature
• Semi-supervised Learning for Few-shot Image-to-Image Translation