Stacked Generative Adversarial Networks
arXiv:1612.04357
Abstract
In this paper, we propose a novel generative model named Stacked Generative Adversarial Networks (SGAN), which is trained to invert the hierarchical representations of a bottom-up discriminative network. Our model consists of a top-down stack of GANs, each learned to generate lower-level representations conditioned on higher-level representations. A representation discriminator is introduced at each feature hierarchy to encourage the representation manifold of the generator to align with that of the bottom-up discriminative network, leveraging the powerful discriminative representations to guide the generative model. In addition, we introduce a conditional loss that encourages the use of conditional information from the layer above, and a novel entropy loss that maximizes a variational lower bound on the conditional entropy of generator outputs. We first train each stack independently, and then train the whole model end-to-end. Unlike the original GAN that uses a single noise vector to represent all the variations, our SGAN decomposes variations into multiple levels and gradually resolves uncertainties in the top-down generative process. Based on visual inspection, Inception scores and visual Turing test, we demonstrate that SGAN is able to generate images of much higher quality than GANs without stacking.
arXiv commentCVPR 2017, camera-ready version
References in corpus (35)
- Conditional Generative Adversarial Nets
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Deep Residual Learning for Image Recognition
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Generative Adversarial Text to Image Synthesis
- FitNets: Hints for Thin Deep Nets
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Conditional Image Generation with PixelCNN Decoders
- Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- Pixel Recurrent Neural Networks
- Going Deeper with Convolutions
- Improved Techniques for Training GANs
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- DRAW: A Recurrent Neural Network For Image Generation
- Energy-based Generative Adversarial Network
- FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
- Adversarially Learned Inference
- CNN Features off-the-shelf: an Astounding Baseline for Recognition
- Autoencoding beyond pixels using a learned similarity metric
- Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
- Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
- Texture Synthesis Using Convolutional Neural Networks
- Generative Moment Matching Networks
- MADE: Masked Autoencoder for Distribution Estimation
- Adversarial Autoencoders
- Stacked What-Where Auto-encoders
- LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
- Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
- Attribute2Image: Conditional Image Generation from Visual Attributes
- Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
- Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification
- Discriminative Regularization for Generative Models
- Scribbler: Controlling Deep Image Synthesis with Sketch and Color
- Cross Modal Distillation for Supervision Transfer
- Calibrating Energy-based Generative Adversarial Networks
Cited by in corpus (31)
- Generative Adversarial Networks: An Overview
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Improved Training of Wasserstein GANs
- Pose Guided Person Image Generation
- An Introduction to Image Synthesis with Generative Adversarial Nets
- Do GANs actually learn the distribution? An empirical study
- Multi-style Generative Network for Real-time Transfer
- Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Network
- Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
- Activation Maximization Generative Adversarial Nets
- AGAN: Towards Automated Design of Generative Adversarial Networks
- Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal
- 3D Human Pose Estimation in the Wild by Adversarial Learning
- Class-Splitting Generative Adversarial Networks
- Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
- On the Effectiveness of Least Squares Generative Adversarial Networks
- Fine-grained Synthesis of Unrestricted Adversarial Examples
- Face Translation between Images and Videos using Identity-aware CycleGAN
- RankGAN: A Maximum Margin Ranking GAN for Generating Faces
- DualSDF: Semantic Shape Manipulation using a Two-Level Representation
- PixelNN: Example-based Image Synthesis
- Stopping GAN Violence: Generative Unadversarial Networks
- Unsupervised Domain Adaptation using Generative Models and Self-ensembling
- Collaborative Sampling in Generative Adversarial Networks
- Wasserstein Introspective Neural Networks
- Updating the generator in PPGN-h with gradients flowing through the encoder
- Semi-supervised mp-MRI Data Synthesis with StitchLayer and Auxiliary Distance Maximization
- Learning to Avoid Errors in GANs by Manipulating Input Spaces
- Linear Discriminant Generative Adversarial Networks
- High Diversity Attribute Guided Face Generation with GANs
- Anonymization of labeled TOF-MRA images for brain vessel segmentation using generative adversarial networks