Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks

Abstract

In this paper we introduce a generative parametric model capable of producing high quality samples of natural images. Our approach uses a cascade of convolutional networks within a Laplacian pyramid framework to generate images in a coarse-to-fine fashion. At each level of the pyramid, a separate generative convnet model is trained using the Generative Adversarial Nets (GAN) approach (Goodfellow et al.). Samples drawn from our model are of significantly higher quality than alternate approaches. In a quantitative assessment by human evaluators, our CIFAR10 samples were mistaken for real images around 40% of the time, compared to 10% for samples drawn from a GAN baseline model. We also show samples from models trained on the higher resolution images of the LSUN scene dataset.

References in corpus (6)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• DRAW: A Recurrent Neural Network For Image Generation
• Learning to Generate Chairs, Tables and Cars with Convolutional Networks

Cited by in corpus (327)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Generative Adversarial Networks: An Overview
• Conditional Image Synthesis With Auxiliary Classifier GANs
• GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification
• Generative Adversarial Network in Medical Imaging: A Review
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• The Perception-Distortion Tradeoff
• Energy-based Generative Adversarial Network
• Generating Videos with Scene Dynamics
• Density estimation using Real NVP
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Understanding Deep Convolutional Networks
• Adversarial Feature Learning
• Adversarially Learned Inference
• Autoencoding beyond pixels using a learned similarity metric
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
• A Survey on Dialogue Systems: Recent Advances and New Frontiers
• Semi-Supervised Learning with Generative Adversarial Networks
• Unsupervised Image-to-Image Translation Networks
• Multimodal Intelligence: Representation Learning, Information Fusion, and Applications
• A note on the evaluation of generative models
• cGANs with Projection Discriminator
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• Adversarial Learning for Semi-Supervised Semantic Segmentation
• Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN
• Deep multi-scale video prediction beyond mean square error
• Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
• Towards Principled Methods for Training Generative Adversarial Networks
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Recent Advances in Convolutional Neural Networks
• A Review on Deep Learning Techniques for Video Prediction
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Multimodal Unsupervised Image-to-Image Translation
• TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
• Generative Adversarial Network for Medical Images (MI-GAN)
• Disentangling factors of variation in deep representations using adversarial training
• Triple Generative Adversarial Nets
• Synthesizing Tabular Data using Generative Adversarial Networks
• Adversarial Learning for Neural Dialogue Generation
• Learning What and Where to Draw
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• GraphGAN: Graph Representation Learning with Generative Adversarial Nets
• Representation Learning by Rotating Your Faces
• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• Quantum generative adversarial network for generating discrete distribution
• AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
• Adversarial Ranking for Language Generation
• An Introduction to Image Synthesis with Generative Adversarial Nets
• A Two-Stage Approach to Few-Shot Learning for Image Recognition
• Least Squares Generative Adversarial Networks
• Deep Identity-aware Transfer of Facial Attributes
• Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis
• Adversarial Networks for Spatial Context-Aware Spectral Image Reconstruction from RGB
• Video-to-Video Synthesis
• Dual Discriminator Generative Adversarial Nets
• Learning to Protect Communications with Adversarial Neural Cryptography
• Generative Visual Manipulation on the Natural Image Manifold
• Variable Rate Image Compression with Recurrent Neural Networks
• Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
• Recent Advances in Deep Learning: An Overview
• Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN
• Real-Time Adaptive Image Compression
• A survey of synthetic data augmentation methods in computer vision
• Coupled Generative Adversarial Networks
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• Censoring Representations with an Adversary
• Age Progression/Regression by Conditional Adversarial Autoencoder
• Super Mario as a String: Platformer Level Generation Via LSTMs
• PixelNet: Representation of the pixels, by the pixels, and for the pixels
• Attribute2Image: Conditional Image Generation from Visual Attributes
• Patch-Based Image Inpainting with Generative Adversarial Networks
• GP-GAN: Towards Realistic High-Resolution Image Blending
• Perceptual Generative Adversarial Networks for Small Object Detection
• Super-Resolution with Deep Convolutional Sufficient Statistics
• Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• Deep Probabilistic Programming
• Unsupervised Learning of Visual Structure using Predictive Generative Networks
• Photographic Image Synthesis with Cascaded Refinement Networks
• ChatPainter: Improving Text to Image Generation using Dialogue
• Data Augmentation in Emotion Classification Using Generative Adversarial Networks
• Triangle Generative Adversarial Networks
• Reconstruction of turbulent data with deep generative models for semantic inpainting from TURB-Rot database
• Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification
• Generating Images from Captions with Attention
• Stabilizing GAN Training with Multiple Random Projections
• Semantic Facial Expression Editing using Autoencoded Flow
• Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks
• On the Effects of Batch and Weight Normalization in Generative Adversarial Networks
• Improved Adversarial Systems for 3D Object Generation and Reconstruction
• CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning
• Improving Variational Autoencoder with Deep Feature Consistent and Generative Adversarial Training
• Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation
• CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training
• Cycle-Consistent Speech Enhancement
• DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
• PixelNet: Towards a General Pixel-level Architecture
• Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks
• Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Network
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
• Label Refinement Network for Coarse-to-Fine Semantic Segmentation
• Generative Adversarial Nets from a Density Ratio Estimation Perspective
• How convolutional neural network see the world - A survey of convolutional neural network visualization methods
• Is Generator Conditioning Causally Related to GAN Performance?
• Discriminative Regularization for Generative Models
• Activation Maximization Generative Adversarial Nets
• Learning Beyond Human Expertise with Generative Models for Dental Restorations
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• Unsupervised Text Style Transfer using Language Models as Discriminators
• Stacked Generative Adversarial Networks
• Towards Large-Pose Face Frontalization in the Wild
• Ensembles of Generative Adversarial Networks
• Generative Models for Low-Rank Video Representation and Reconstruction
• Retinal Vasculature Segmentation Using Local Saliency Maps and Generative Adversarial Networks For Image Super Resolution
• Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets
• Learning Hierarchical Features from Generative Models
• Conditional Generative Moment-Matching Networks
• Generating Images Part by Part with Composite Generative Adversarial Networks
• Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks
• Temporal Generative Adversarial Nets with Singular Value Clipping
• Early Action Prediction with Generative Adversarial Networks
• 3D Human Pose Estimation in the Wild by Adversarial Learning
• Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring
• Multi-view Generative Adversarial Networks
• Efficient inference in occlusion-aware generative models of images
• Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
• Distributionally Robust Chance Constrained Programming with Generative Adversarial Networks (GANs)
• Creativity: Generating Diverse Questions using Variational Autoencoders
• Generative Adversarial Parallelization
• Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
• Landmark Assisted CycleGAN for Cartoon Face Generation
• Generative Adversarial Talking Head: Bringing Portraits to Life with a Weakly Supervised Neural Network
• Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion
• A Good Practice Towards Top Performance of Face Recognition: Transferred Deep Feature Fusion
• Sketch-based Creativity Support Tools using Deep Learning
• Visual Intelligence through Human Interaction
• High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis
• Fundus2Angio: A Conditional GAN Architecture for Generating Fluorescein Angiography Images from Retinal Fundus Photography
• Video Imagination from a Single Image with Transformation Generation
• Deep Back-Projection Networks For Super-Resolution
• Diversified Texture Synthesis with Feed-forward Networks
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• Shape Inpainting using 3D Generative Adversarial Network and Recurrent Convolutional Networks
• Comparative Study on Generative Adversarial Networks
• MSG-GAN: Multi-Scale Gradients for Generative Adversarial Networks
• FCC-GAN: A Fully Connected and Convolutional Net Architecture for GANs
• Skill Rating for Generative Models
• Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation
• Learning to Sketch with Deep Q Networks and Demonstrated Strokes
• Learning Kernel for Conditional Moment-Matching Discrepancy-based Image Classification
• Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
• Attacks on State-of-the-Art Face Recognition using Attentional Adversarial Attack Generative Network
• High-Quality Face Image SR Using Conditional Generative Adversarial Networks
• Bandwidth Extension on Raw Audio via Generative Adversarial Networks
• Diffusion-based Document Layout Generation
• Global-Local Face Upsampling Network
• Transferring GANs: generating images from limited data
• Generative adversarial networks and adversarial methods in biomedical image analysis
• Optical Flow Estimation using a Spatial Pyramid Network
• Synthesizing Filamentary Structured Images with GANs
• AC-BLSTM: Asymmetric Convolutional Bidirectional LSTM Networks for Text Classification
• Towards Principled Unsupervised Learning
• 3D Shape Induction from 2D Views of Multiple Objects
• Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks
• eCommerceGAN : A Generative Adversarial Network for E-commerce
• Conditional Image-to-Image Translation
• Adversarial Information Factorization
• Identity-preserving Face Recovery from Stylized Portraits
• Relaxed Wasserstein with Applications to GANs
• Deep Illumination: Approximating Dynamic Global Illumination with Generative Adversarial Network
• Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
• Quantum Generative Adversarial Networks For Anomaly Detection In High Energy Physics
• PTNet: A High-Resolution Infant MRI Synthesizer Based on Transformer
• Joint Multi-Scale Tone Mapping and Denoising for HDR Image Enhancement
• DeepMovie: Using Optical Flow and Deep Neural Networks to Stylize Movies
• ArtGAN: Artwork Synthesis with Conditional Categorical GANs
• A Review of Semi Supervised Learning Theories and Recent Advances
• Hierarchical Adversarially Learned Inference
• Energy-based Models for Video Anomaly Detection
• Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
• Every Smile is Unique: Landmark-Guided Diverse Smile Generation
• On the Effectiveness of Least Squares Generative Adversarial Networks
• Photorealistic Facial Expression Synthesis by the Conditional Difference Adversarial Autoencoder
• Learning stochastic object models from medical imaging measurements by use of advanced ambient generative adversarial networks
• Biased Mixtures Of Experts: Enabling Computer Vision Inference Under Data Transfer Limitations
• CANZSL: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language
• Learning Typographic Style
• Learning to Generate with Memory
• Sequential IoT Data Augmentation using Generative Adversarial Networks
• A Novel Convolutional Neural Network for Image Steganalysis with Shared Normalization
• DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data
• To Create What You Tell: Generating Videos from Captions
• Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks
• UGAN: Untraceable GAN for Multi-Domain Face Translation
• Improving Video Generation for Multi-functional Applications
• Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
• Neural Networks for Modeling Source Code Edits
• Geodesic Clustering in Deep Generative Models
• Hierarchical Patch VAE-GAN: Generating Diverse Videos from a Single Sample
• Hyperparameter Optimization of Generative Adversarial Network Models for High-Energy Physics Simulations
• Semantically Consistent Image Completion with Fine-grained Details
• cGANs with Multi-Hinge Loss
• IterGANs: Iterative GANs to Learn and Control 3D Object Transformation
• Learning Residual Images for Face Attribute Manipulation
• Adversarial Learning for Chinese NER from Crowd Annotations
• GILT: Generating Images from Long Text
• Spatial Frequency Loss for Learning Convolutional Autoencoders
• DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation
• Lightweight Pyramid Networks for Image Deraining
• Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders
• Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer
• Learning High Dynamic Range from Outdoor Panoramas
• A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis
• Fine-grained Synthesis of Unrestricted Adversarial Examples
• Annealed Generative Adversarial Networks
• Associative Adversarial Networks
• The Neural Painter: Multi-Turn Image Generation
• Towards Automatic Image Editing: Learning to See another You
• Synthesizing Dynamic Patterns by Spatial-Temporal Generative ConvNet
• CounteRGAN: Generating Realistic Counterfactuals with Residual Generative Adversarial Nets
• Multi-Scale Video Frame-Synthesis Network with Transitive Consistency Loss
• Points2Pix: 3D Point-Cloud to Image Translation using conditional Generative Adversarial Networks
• Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
• A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection
• Deep Cross-Modal Audio-Visual Generation
• Deep Tensor Convolution on Multicores
• RankGAN: A Maximum Margin Ranking GAN for Generating Faces
• Towards Proof Synthesis Guided by Neural Machine Translation for Intuitionistic Propositional Logic
• JGAN: A Joint Formulation of GAN for Synthesizing Images and Labels
• What Is It Like Down There? Generating Dense Ground-Level Views and Image Features From Overhead Imagery Using Conditional Generative Adversarial Networks
• Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds
• PixelNN: Example-based Image Synthesis
• One-to-Many Network for Visually Pleasing Compression Artifacts Reduction
• TreeGAN: Syntax-Aware Sequence Generation with Generative Adversarial Networks
• Towards Better User Studies in Computer Graphics and Vision
• PixelCNN Models with Auxiliary Variables for Natural Image Modeling
• Can GAN Learn Topological Features of a Graph?
• Smart, Sparse Contours to Represent and Edit Images
• Hybrid VAE: Improving Deep Generative Models using Partial Observations
• Stopping GAN Violence: Generative Unadversarial Networks
• Teaching Machines to Converse
• ACGAN-based Data Augmentation Integrated with Long-term Scalogram for Acoustic Scene Classification
• Voice Conversion using Convolutional Neural Networks
• Active Decision Boundary Annotation with Deep Generative Models
• Deep Learning for Mean Field Games with non-separable Hamiltonians
• Flexible and Scalable Deep Learning with MMLSpark
• Learning Temporal Transformations From Time-Lapse Videos
• Visual Decoding of Targets During Visual Search From Human Eye Fixations
• Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis
• Pipeline Generative Adversarial Networks for Facial Images Generation with Multiple Attributes
• Generative Collaborative Networks for Single Image Super-Resolution
• Learning Disentangled Representations of Satellite Image Time Series
• A Mathematical Framework for Learning Probability Distributions
• Multimodal Image-to-Image Translation via Mutual Information Estimation and Maximization
• Future Urban Scenes Generation Through Vehicles Synthesis
• Predicting Real-Time Locational Marginal Prices: A GAN-Based Video Prediction Approach
• Mode matching in GANs through latent space learning and inversion
• CapsGAN: Using Dynamic Routing for Generative Adversarial Networks
• Longitudinal Face Aging in the Wild - Recent Deep Learning Approaches
• Text Generation Based on Generative Adversarial Nets with Latent Variable
• Patch Correspondences for Interpreting Pixel-level CNNs
• Long-Term Image Boundary Prediction
• Image2GIF: Generating Cinemagraphs using Recurrent Deep Q-Networks
• Generative Image Modeling using Style and Structure Adversarial Networks
• Generative Mixture of Networks
• Learning Generative Models of Tissue Organization with Supervised GANs
• How to make a pizza: Learning a compositional layer-based GAN model
• Parametric Synthesis of Text on Stylized Backgrounds using PGGANs
• Applying Generative Adversarial Networks to Intelligent Subsurface Imaging and Identification
• DISCO Nets: DISsimilarity COefficient Networks
• Evolvement Constrained Adversarial Learning for Video Style Transfer
• Omni-GAN: On the Secrets of cGANs and Beyond
• Dense 3D Point Cloud Reconstruction Using a Deep Pyramid Network
• Dynamic Video Segmentation Network
• GMM-Based Generative Adversarial Encoder Learning
• Symmetries and control in generative neural nets
• Max-Margin Deep Generative Models for (Semi-)Supervised Learning
• Portrait Shadow Manipulation
• Adversarial Learning of Task-Oriented Neural Dialog Models
• Stable Distribution Alignment Using the Dual of the Adversarial Distance
• xAI-CycleGAN, a Cycle-Consistent Generative Assistive Network
• Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey
• How good is my GAN?
• Adversarial point set registration
• Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling
• Learning to Avoid Errors in GANs by Manipulating Input Spaces
• Flipped-Adversarial AutoEncoders
• Generative Creativity: Adversarial Learning for Bionic Design
• Excavate Condition-invariant Space by Intrinsic Encoder
• Deep Markov Random Field for Image Modeling
• Spatio-Temporal Image Boundary Extrapolation
• Regularized Sequential Latent Variable Models with Adversarial Neural Networks
• Making a Case for Learning Motion Representations with Phase
• Generative Adversarial Networks with Inverse Transformation Unit
• Variational Autoencoded Regression: High Dimensional Regression of Visual Data on Complex Manifold
• HAD-GAN: A Human-perception Auxiliary Defense GAN to Defend Adversarial Examples
• Forget the Learning Rate, Decay Loss
• Fine-scale Surface Normal Estimation using a Single NIR Image
• Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks
• Enhanced Magnetic Resonance Image Synthesis with Contrast-Aware Generative Adversarial Networks
• Generative Adversarial Network: Some Analytical Perspectives
• Unsupervised Meta-learning of Figure-Ground Segmentation via Imitating Visual Effects
• Adversarial Code Learning for Image Generation
• Multi-Task Generative Adversarial Nets with Shared Memory for Cross-Domain Coordination Control
• A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation
• A Study of Cross-domain Generative Models applied to Cartoon Series
• Enhancing audio quality for expressive Neural Text-to-Speech
• Replicating Active Appearance Model by Generator Network
• StackGAN: Facial Image Generation Optimizations
• LR-to-HR Face Hallucination with an Adversarial Progressive Attribute-Induced Network
• 3D-MOV: Audio-Visual LSTM Autoencoder for 3D Reconstruction of Multiple Objects from Video
• Authoring image decompositions with generative models
• Deep Semantic Hashing with Generative Adversarial Networks
• NeurInt : Learning to Interpolate through Neural ODEs
• Selective Sampling and Mixture Models in Generative Adversarial Networks
• Unsupervised Representation Learning with Laplacian Pyramid Auto-encoders
• A Two-Step Disentanglement Method
• Deep Convolutional Generative Adversarial Network Based Food Recognition Using Partially Labeled Data
• Virtual Conditional Generative Adversarial Networks
• Generating new pictures in complex datasets with a simple neural network
• KGAN: How to Break The Minimax Game in GAN
• Image Disguise based on Generative Model