Stacked Generative Adversarial Networks

Abstract

In this paper, we propose a novel generative model named Stacked Generative Adversarial Networks (SGAN), which is trained to invert the hierarchical representations of a bottom-up discriminative network. Our model consists of a top-down stack of GANs, each learned to generate lower-level representations conditioned on higher-level representations. A representation discriminator is introduced at each feature hierarchy to encourage the representation manifold of the generator to align with that of the bottom-up discriminative network, leveraging the powerful discriminative representations to guide the generative model. In addition, we introduce a conditional loss that encourages the use of conditional information from the layer above, and a novel entropy loss that maximizes a variational lower bound on the conditional entropy of generator outputs. We first train each stack independently, and then train the whole model end-to-end. Unlike the original GAN that uses a single noise vector to represent all the variations, our SGAN decomposes variations into multiple levels and gradually resolves uncertainties in the top-down generative process. Based on visual inspection, Inception scores and visual Turing test, we demonstrate that SGAN is able to generate images of much higher quality than GANs without stacking.

CVPR 2017, camera-ready version

References in corpus (13)

• Conditional Generative Adversarial Nets
• FitNets: Hints for Thin Deep Nets
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Going Deeper with Convolutions
• DRAW: A Recurrent Neural Network For Image Generation
• Energy-based Generative Adversarial Network
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Generative Moment Matching Networks
• MADE: Masked Autoencoder for Distribution Estimation
• LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
• Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
• Scribbler: Controlling Deep Image Synthesis with Sketch and Color
• Calibrating Energy-based Generative Adversarial Networks

Cited by in corpus (31)

• Generative Adversarial Networks: An Overview
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Improved Training of Wasserstein GANs
• Pose Guided Person Image Generation
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Do GANs actually learn the distribution? An empirical study
• Multi-style Generative Network for Real-time Transfer
• Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Network
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
• Activation Maximization Generative Adversarial Nets
• AGAN: Towards Automated Design of Generative Adversarial Networks
• Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal
• 3D Human Pose Estimation in the Wild by Adversarial Learning
• Class-Splitting Generative Adversarial Networks
• Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
• On the Effectiveness of Least Squares Generative Adversarial Networks
• Fine-grained Synthesis of Unrestricted Adversarial Examples
• Face Translation between Images and Videos using Identity-aware CycleGAN
• RankGAN: A Maximum Margin Ranking GAN for Generating Faces
• DualSDF: Semantic Shape Manipulation using a Two-Level Representation
• PixelNN: Example-based Image Synthesis
• Stopping GAN Violence: Generative Unadversarial Networks
• Unsupervised Domain Adaptation using Generative Models and Self-ensembling
• Collaborative Sampling in Generative Adversarial Networks
• Wasserstein Introspective Neural Networks
• Updating the generator in PPGN-h with gradients flowing through the encoder
• Semi-supervised mp-MRI Data Synthesis with StitchLayer and Auxiliary Distance Maximization
• Learning to Avoid Errors in GANs by Manipulating Input Spaces
• Linear Discriminant Generative Adversarial Networks
• High Diversity Attribute Guided Face Generation with GANs
• Anonymization of labeled TOF-MRA images for brain vessel segmentation using generative adversarial networks