Energy-based Generative Adversarial Network

Abstract

We introduce the "Energy-based Generative Adversarial Network" model (EBGAN) which views the discriminator as an energy function that attributes low energies to the regions near the data manifold and higher energies to other regions. Similar to the probabilistic GANs, a generator is seen as being trained to produce contrastive samples with minimal energies, while the discriminator is trained to assign high energies to these generated samples. Viewing the discriminator as an energy function allows to use a wide variety of architectures and loss functionals in addition to the usual binary classifier with logistic output. Among them, we show one instantiation of EBGAN framework as using an auto-encoder architecture, with the energy being the reconstruction error, in place of the discriminator. We show that this form of EBGAN exhibits more stable behavior than regular GANs during training. We also show that a single-scale architecture can be trained to generate high-resolution images.

Submitted to ICLR 2017

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Matching Networks for One Shot Learning
• Autoencoding beyond pixels using a learned similarity metric
• Stacked What-Where Auto-encoders
• Deep Directed Generative Models with Energy-Based Probability Estimation

Cited by in corpus (293)

• Generative Adversarial Networks: An Overview
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Generative Adversarial Network in Medical Imaging: A Review
• Toward Multimodal Image-to-Image Translation
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Deep Generative Design: Integration of Topology Optimization and Generative Models
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Perceptual Adversarial Networks for Image-to-Image Transformation
• U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Learning Graph Embedding with Adversarial Training Methods
• Differentially Private Generative Adversarial Network
• Multimodal Unsupervised Image-to-Image Translation
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Identification of Deep Network Generated Images Using Disparities in Color Components
• Image De-raining Using a Conditional Generative Adversarial Network
• Microstructural Materials Design via Deep Adversarial Learning Methodology
• An Overview of Deep Semi-Supervised Learning
• Mode Regularized Generative Adversarial Networks
• On Convergence and Stability of GANs
• Good Semi-supervised Learning that Requires a Bad GAN
• An empirical study on evaluation metrics of generative adversarial networks
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Optimizing the Latent Space of Generative Networks
• Wasserstein Auto-Encoders
• Learning in Implicit Generative Models
• Data-driven Machinery Fault Diagnosis: A Comprehensive Review
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Least Squares Generative Adversarial Networks
• Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
• Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
• Generative Adversarial Networks (GANs) in Networking: A Comprehensive Survey & Evaluation
• Adversarial Feature Matching for Text Generation
• Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
• Enhanced Balancing GAN: Minority-class Image Generation
• Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis
• Diffusion Model-Based Image Editing: A Survey
• Age Progression/Regression by Conditional Adversarial Autoencoder
• Generative Models for Automatic Chemical Design
• Implicit Generation and Generalization in Energy-Based Models
• Connecting Generative Adversarial Networks and Actor-Critic Methods
• On the Origin of Deep Learning
• Paraphrasing Complex Network: Network Compression via Factor Transfer
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
• SteganoGAN: High Capacity Image Steganography with GANs
• Triangle Generative Adversarial Networks
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• Stabilizing GAN Training with Multiple Random Projections
• Adversarially Regularized Graph Autoencoder for Graph Embedding
• Adversarial Network Embedding
• On the Effects of Batch and Weight Normalization in Generative Adversarial Networks
• Maximum Entropy Generators for Energy-Based Models
• FrankenGAN: Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANs
• Implicit Maximum Likelihood Estimation
• AccEar: Accelerometer Acoustic Eavesdropping with Unconstrained Vocabulary
• DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
• Spatial Evolutionary Generative Adversarial Networks
• MAGAN: Margin Adaptation for Generative Adversarial Networks
• Multi-Generator Generative Adversarial Nets
• Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling
• Dropout-GAN: Learning from a Dynamic Ensemble of Discriminators
• Activation Maximization Generative Adversarial Nets
• Learning Beyond Human Expertise with Generative Models for Dental Restorations
• Unsupervised Text Style Transfer using Language Models as Discriminators
• Stacked Generative Adversarial Networks
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks
• Denoising Autoencoders for Overgeneralization in Neural Networks
• An Improved Evaluation Framework for Generative Adversarial Networks
• ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
• BourGAN: Generative Networks with Metric Embeddings
• A Convex Duality Framework for GANs
• Adversarial Symmetric Variational Autoencoder
• Neural Face Editing with Intrinsic Image Disentangling
• Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives
• Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments
• DiverGAN: An Efficient and Effective Single-Stage Framework for Diverse Text-to-Image Generation
• Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
• Class-Splitting Generative Adversarial Networks
• Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields
• Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligence
• Improved Training of Generative Adversarial Networks Using Representative Features
• Adversarial Domain Adaptation for Stable Brain-Machine Interfaces
• Modeling documents with Generative Adversarial Networks
• Generative Adversarial Networks for Financial Trading Strategies Fine-Tuning and Combination
• High-resolution Deep Convolutional Generative Adversarial Networks
• Global Convergence to the Equilibrium of GANs using Variational Inequalities
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• FCC-GAN: A Fully Connected and Convolutional Net Architecture for GANs
• Embedding Deep Networks into Visual Explanations
• High-Quality Face Image SR Using Conditional Generative Adversarial Networks
• Exploiting the Generative Adversarial Network Approach to Create a Synthetic Topography Corneal Image
• Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
• Image Clustering using an Augmented Generative Adversarial Network and Information Maximization
• QGAN: Quantized Generative Adversarial Networks
• Relaxed Wasserstein with Applications to GANs
• eCommerceGAN : A Generative Adversarial Network for E-commerce
• Image-to-Image Translation: Methods and Applications
• An Adversarial Regularisation for Semi-Supervised Training of Structured Output Neural Networks
• Generative Adversarial Networks as Variational Training of Energy Based Models
• PathGAN: Visual Scanpath Prediction with Generative Adversarial Networks
• Adversarial Network Bottleneck Features for Noise Robust Speaker Verification
• Adversarial Contrastive Estimation
• GANGs: Generative Adversarial Network Games
• Robust deep learning for eye fundus images: Bridging real and synthetic data for enhancing generalization
• Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning
• Symmetric Variational Autoencoder and Connections to Adversarial Learning
• Deep UL2DL: Channel Knowledge Transfer from Uplink to Downlink
• TorchGAN: A Flexible Framework for GAN Training and Evaluation
• On the Effectiveness of Least Squares Generative Adversarial Networks
• Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics
• Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation
• Learning Energy-Based Models by Diffusion Recovery Likelihood
• CERN: Confidence-Energy Recurrent Network for Group Activity Recognition
• Learning Deep Energy Models: Contrastive Divergence vs. Amortized MLE
• Real or Not Real, that is the Question
• CANZSL: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language
• Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guarantees
• Unbalanced GANs: Pre-training the Generator of Generative Adversarial Network using Variational Autoencoder
• Quantitatively Evaluating GANs With Divergences Proposed for Training
• DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data
• Improving Video Generation for Multi-functional Applications
• Rare Disease Detection by Sequence Modeling with Generative Adversarial Networks
• Task Specific Adversarial Cost Function
• UGAN: Untraceable GAN for Multi-Domain Face Translation
• Maximizing Discrimination Capability of Knowledge Distillation with Energy Function
• Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localization
• SPLAT: Semantic Pixel-Level Adaptation Transforms for Detection
• Energy-Based Imitation Learning
• Recent Progress of Face Image Synthesis
• Ways of Conditioning Generative Adversarial Networks
• Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
• TequilaGAN: How to easily identify GAN samples
• Distributional Adversarial Networks
• One-Class Adversarial Nets for Fraud Detection
• DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)
• Deep Co-Space: Sample Mining Across Feature Transformation for Semi-Supervised Learning
• Game of Learning Bloch Equation Simulations for MR Fingerprinting
• Controllable Generative Adversarial Network
• Proximal Alternating Direction Network: A Globally Converged Deep Unrolling Framework
• Versatile Auxiliary Classifier with Generative Adversarial Network (VAC+GAN), Multi Class Scenarios
• Reducing Uncertainty in Undersampled MRI Reconstruction with Active Acquisition
• Differential Generative Adversarial Networks: Synthesizing Non-linear Facial Variations with Limited Number of Training Data
• Improving GAN Training with Probability Ratio Clipping and Sample Reweighting
• Synergetic Learning Systems: Concept, Architecture, and Algorithms
• Annealing Genetic GAN for Minority Oversampling
• Boltzmann machines and energy-based models
• Towards Stable and Comprehensive Domain Alignment: Max-Margin Domain-Adversarial Training
• TripletGAN: Training Generative Model with Triplet Loss
• Annealed Generative Adversarial Networks
• Imbalanced Data Learning by Minority Class Augmentation using Capsule Adversarial Networks
• GAN- vs. JPEG2000 Image Compression for Distributed Automotive Perception: Higher Peak SNR Does Not Mean Better Semantic Segmentation
• Inception Score, Label Smoothing, Gradient Vanishing and -log(D(x)) Alternative
• Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling by Exploring Energy of the Discriminator
• Face Translation between Images and Videos using Identity-aware CycleGAN
• SLA Decomposition for Network Slicing: A Deep Neural Network Approach
• Learning Spatial Pyramid Attentive Pooling in Image Synthesis and Image-to-Image Translation
• MCL-GAN: Generative Adversarial Networks with Multiple Specialized Discriminators
• Rates of convergence for density estimation with generative adversarial networks
• CounteRGAN: Generating Realistic Counterfactuals with Residual Generative Adversarial Nets
• Recent Advances of Image Steganography with Generative Adversarial Networks
• Energy-Based Sequence GANs for Recommendation and Their Connection to Imitation Learning
• Deep Learning Based Computed Tomography Whys and Wherefores
• Improving GAN Training via Binarized Representation Entropy (BRE) Regularization
• Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets
• Label-Noise Robust Generative Adversarial Networks
• Exploring global diverse attention via pairwise temporal relation for video summarization
• Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastings
• VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
• RankGAN: A Maximum Margin Ranking GAN for Generating Faces
• DeepMasterPrints: Generating MasterPrints for Dictionary Attacks via Latent Variable Evolution
• RL-GAN-Net: A Reinforcement Learning Agent Controlled GAN Network for Real-Time Point Cloud Shape Completion
• On Scalable and Efficient Computation of Large Scale Optimal Transport
• Deep learning approach in multi-scale prediction of turbulent mixing-layer
• Constrained Generative Adversarial Network Ensembles for Sharable Synthetic Data Generation
• Deep Stacked Networks with Residual Polishing for Image Inpainting
• Metric Learning-based Generative Adversarial Network
• Adversarial Colorization Of Icons Based On Structure And Color Conditions
• Generative Adversarial Frontal View to Bird View Synthesis
• On the Need for Topology-Aware Generative Models for Manifold-Based Defenses
• Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning
• Image Quality Assessment Techniques Show Improved Training and Evaluation of Autoencoder Generative Adversarial Networks
• xAI-GAN: Enhancing Generative Adversarial Networks via Explainable AI Systems
• Designing GANs: A Likelihood Ratio Approach
• Probabilistic Generative Adversarial Networks
• Natural and Realistic Single Image Super-Resolution with Explicit Natural Manifold Discrimination
• Supervised Adversarial Networks for Image Saliency Detection
• Unpaired Photo-to-Caricature Translation on Faces in the Wild
• Acoustic Feature Learning via Deep Variational Canonical Correlation Analysis
• Can GAN Learn Topological Features of a Graph?
• Improving Direct Physical Properties Prediction of Heterogeneous Materials from Imaging Data via Convolutional Neural Network and a Morphology-Aware Generative Model
• Max-Affine Spline Insights into Deep Generative Networks
• IAN: Combining Generative Adversarial Networks for Imaginative Face Generation
• Deep Generative Learning via Variational Gradient Flow
• DISSECT: Disentangled Simultaneous Explanations via Concept Traversals
• Capturing Variabilities from Computed Tomography Images with Generative Adversarial Networks
• GO Gradient for Expectation-Based Objectives
• FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery
• Generative Adversarial Forests for Better Conditioned Adversarial Learning
• A Mathematical Framework for Learning Probability Distributions
• BasketballGAN: Generating Basketball Play Simulation Through Sketching
• Joint Training of Variational Auto-Encoder and Latent Energy-Based Model
• Continuous Transition: Improving Sample Efficiency for Continuous Control Problems via MixUp
• Image-to-Image Translation with Low Resolution Conditioning
• Segmentation Guided Image-to-Image Translation with Adversarial Networks
• Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Model
• Inclusive GAN: Improving Data and Minority Coverage in Generative Models
• Landmark Detection in Low Resolution Faces with Semi-Supervised Learning
• Learning Disentangled Representations of Satellite Image Time Series
• Class-Distinct and Class-Mutual Image Generation with GANs
• Versatile Auxiliary Classifier with Generative Adversarial Network (VAC+GAN)
• Depth Structure Preserving Scene Image Generation
• Towards Grounding Conceptual Spaces in Neural Representations
• Reconstruction of Simulation-Based Physical Field by Reconstruction Neural Network Method
• Face Aging with Contextual Generative Adversarial Nets
• Manifold-valued Image Generation with Wasserstein Generative Adversarial Nets
• Generative Adversarial Mapping Networks
• Pipeline Generative Adversarial Networks for Facial Images Generation with Multiple Attributes
• Tempered Adversarial Networks
• GANs with Conditional Independence Graphs: On Subadditivity of Probability Divergences
• Learning Discriminators as Energy Networks in Adversarial Learning
• A Novel Framework for Selection of GANs for an Application
• Robust GANs against Dishonest Adversaries
• Risk Bounds for Unsupervised Cross-Domain Mapping with IPMs
• Introspective Generative Modeling: Decide Discriminatively
• Kernel-Guided Training of Implicit Generative Models with Stability Guarantees
• Set Distribution Networks: a Generative Model for Sets of Images
• General Probabilistic Surface Optimization and Log Density Estimation
• More Information Supervised Probabilistic Deep Face Embedding Learning
• TinyGAN: Distilling BigGAN for Conditional Image Generation
• Scyclone: High-Quality and Parallel-Data-Free Voice Conversion Using Spectrogram and Cycle-Consistent Adversarial Networks
• Auto-Embedding Generative Adversarial Networks for High Resolution Image Synthesis
• Deep Consensus Learning
• Manifold-preserved GANs
• "Best-of-Many-Samples" Distribution Matching
• Inferential Wasserstein Generative Adversarial Networks
• Semi-Supervised Learning via New Deep Network Inversion
• Alleviation of Gradient Exploding in GANs: Fake Can Be Real
• MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining
• Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks
• GANE: A Generative Adversarial Network Embedding
• Generative Creativity: Adversarial Learning for Bionic Design
• Improving Joint Training of Inference Networks and Structured Prediction Energy Networks
• Anomaly Detection by One Class Latent Regularized Networks
• Stabilizing Deep Tomographic Reconstruction
• Autoencoding Under Normalization Constraints
• Provably Convergent Algorithms for Solving Inverse Problems Using Generative Models
• Adversarial sampling of unknown and high-dimensional conditional distributions
• GANILLA: Generative Adversarial Networks for Image to Illustration Translation
• Restricting Greed in Training of Generative Adversarial Network
• Fast Ensemble Learning Using Adversarially-Generated Restricted Boltzmann Machines
• On reproduction of On the regularization of Wasserstein GANs
• Reparameterized Variational Divergence Minimization for Stable Imitation
• Online Kernel based Generative Adversarial Networks
• Generative networks as inverse problems with fractional wavelet scattering networks
• Unsupervised Domain Alignment to Mitigate Low Level Dataset Biases
• A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models
• Deep PDF: Probabilistic Surface Optimization and Density Estimation
• Visual Data Augmentation through Learning
• Instance Map based Image Synthesis with a Denoising Generative Adversarial Network
• Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP
• AIP: Adversarial Iterative Pruning Based on Knowledge Transfer for Convolutional Neural Networks
• Machine Vision for Improved Human-Robot Cooperation in Adverse Underwater Conditions
• Dist-GAN: An Improved GAN using Distance Constraints
• Dual Encoder-Decoder based Generative Adversarial Networks for Disentangled Facial Representation Learning
• LR-to-HR Face Hallucination with an Adversarial Progressive Attribute-Induced Network
• Pre-Training Transformers as Energy-Based Cloze Models
• Learning to Inpaint by Progressively Growing the Mask Regions
• Improving Stability of LS-GANs for Audio and Speech Signals
• DINO: A Conditional Energy-Based GAN for Domain Translation
• SetGAN: Improving the stability and diversity of generative models through a permutation invariant architecture
• CANet: Cross-disease Attention Network for Joint Diabetic Retinopathy and Diabetic Macular Edema Grading
• Learning relevant features for statistical inference
• From Thumbnails to Summaries - A single Deep Neural Network to Rule Them All
• CerfGAN: A Compact, Effective, Robust, and Fast Model for Unsupervised Multi-Domain Image-to-Image Translation
• Forget the Learning Rate, Decay Loss
• Linear Discriminant Generative Adversarial Networks
• DefogGAN: Predicting Hidden Information in the StarCraft Fog of War with Generative Adversarial Nets
• Reducing the Representation Error of GAN Image Priors Using the Deep Decoder
• Human Annotations Improve GAN Performances
• MMCGAN: Generative Adversarial Network with Explicit Manifold Prior
• Brain-Inspired Inference on Missing Video Sequence
• An Interpretable Generative Model for Handwritten Digit Image Synthesis
• ReshapeGAN: Object Reshaping by Providing A Single Reference Image
• Generating Videos of Zero-Shot Compositions of Actions and Objects
• Adversarially-Trained Normalized Noisy-Feature Auto-Encoder for Text Generation
• Generative Adversarial Networks with Inverse Transformation Unit
• Adversarial Code Learning for Image Generation
• Escaping from Collapsing Modes in a Constrained Space
• An Efficient Style Virtual Try on Network for Clothing Business Industry
• A Survey on Adversarial Image Synthesis