How Generative Adversarial Networks and Their Variants Work: An Overview

Abstract

Generative Adversarial Networks (GAN) have received wide attention in the machine learning field for their potential to learn high-dimensional, complex real data distribution. Specifically, they do not rely on any assumptions about the distribution and can generate real-like samples from latent space in a simple manner. This powerful property leads GAN to be applied to various applications such as image synthesis, image attribute editing, image translation, domain adaptation and other academic fields. In this paper, we aim to discuss the details of GAN for those readers who are familiar with, but do not comprehend GAN deeply or who wish to view GAN from various perspectives. In addition, we explain how GAN operates and the fundamental meaning of various objective functions that have been suggested recently. We then focus on how the GAN can be combined with an autoencoder framework. Finally, we enumerate the GAN variants that are applied to various tasks and other fields for those who are interested in exploiting GAN for their research.

41 pages, 16 figures, Published in ACM Computing Surveys (CSUR)

References in corpus (79)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Continuous control with deep reinforcement learning
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• WaveNet: A Generative Model for Raw Audio
• Spectral Normalization for Generative Adversarial Networks
• Trust Region Policy Optimization
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Pixel Recurrent Neural Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Tutorial on Variational Autoencoders
• NIPS 2016 Tutorial: Generative Adversarial Networks
• A guide to convolution arithmetic for deep learning
• Energy-based Generative Adversarial Network
• Generating Videos with Scene Dynamics
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Adversarial Feature Learning
• Adversarially Learned Inference
• Autoencoding beyond pixels using a learned similarity metric
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Invertible Conditional GANs for image editing
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models
• A DIRT-T Approach to Unsupervised Domain Adaptation
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• cGANs with Projection Discriminator
• Unrolled Generative Adversarial Networks
• Perceptual Adversarial Networks for Image-to-Image Transformation
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• Generative Moment Matching Networks
• Generative Adversarial Imitation Learning
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Mode Regularized Generative Adversarial Networks
• Triple Generative Adversarial Nets
• A Note on the Inception Score
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• Continual Learning with Deep Generative Replay
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Representation Learning by Rotating Your Faces
• Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
• Adversarial Ranking for Language Generation
• Least Squares Generative Adversarial Networks
• Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks
• Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
• Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
• Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks
• Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis
• Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
• TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network
• Boundary-Seeking Generative Adversarial Networks
• Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
• On integral probability metrics, ϕ-divergences and binary classification
• Connecting Generative Adversarial Networks and Actor-Critic Methods
• Perceptual Generative Adversarial Networks for Small Object Detection
• PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications
• CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training
• One-Sided Unsupervised Domain Mapping
• MAGAN: Margin Adaptation for Generative Adversarial Networks
• SSGAN: Secure Steganography Based on Generative Adversarial Networks
• GeneGAN: Learning Object Transfiguration and Attribute Subspace from Unpaired Data
• Face Aging With Conditional Generative Adversarial Networks
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs
• Semi-Latent GAN: Learning to generate and modify facial images from attributes
• Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
• Unsupervised Visual Attribute Transfer with Reconfigurable Generative Adversarial Networks
• Unsupervised Cipher Cracking Using Discrete GANs
• Generating images with recurrent adversarial networks
• Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
• An Online Learning Approach to Generative Adversarial Networks
• Image Generation and Editing with Variational Info Generative AdversarialNetworks
• Energy-Based Sequence GANs for Recommendation and Their Connection to Imitation Learning
• Domain Adaptation Using Adversarial Learning for Autonomous Navigation
• SeGAN: Segmenting and Generating the Invisible

Cited by in corpus (37)

• The relativistic discriminator: a key element missing from standard GAN
• Monocular Depth Estimation Based On Deep Learning: An Overview
• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Adversarial Text-to-Image Synthesis: A Review
• Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
• Deepfake Video Detection Using Convolutional Vision Transformer
• Generative Adversarial Networks for Spatio-temporal Data: A Survey
• Learning and Inference on Generative Adversarial Quantum Circuits
• Attacking Recommender Systems with Augmented User Profiles
• Stabilizing Generative Adversarial Networks: A Survey
• Recent Developments Combining Ensemble Smoother and Deep Generative Networks for Facies History Matching
• Shilling Black-box Recommender Systems by Learning to Generate Fake User Profiles
• Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments
• Generative Adversarial Networks: A Survey Towards Private and Secure Applications
• BSDM: Background Suppression Diffusion Model for Hyperspectral Anomaly Detection
• A Manifesto for Modern Fog and Edge Computing: Vision, New Paradigms, Opportunities, and Future Directions
• From MIM-Based GAN to Anomaly Detection:Event Probability Influence on Generative Adversarial Networks
• A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis
• MITNet: GAN Enhanced Magnetic Induction Tomography Based on Complex CNN
• GAN-enhanced Conditional Echocardiogram Generation
• Generative adversarial networks for generation and classification of physical rehabilitation movement episodes
• Data-Driven Modeling of Noise Time Series with Convolutional Generative Adversarial Networks
• Hierarchical Mixtures of Generators for Adversarial Learning
• Feasibility of Modeling Orthogonal Frequency-Division Multiplexing Communication Signals with Unsupervised Generative Adversarial Networks
• Training Generative Adversarial Networks with Adaptive Composite Gradient
• Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey
• Generative networks as inverse problems with fractional wavelet scattering networks
• A Novel Measure to Evaluate Generative Adversarial Networks Based on Direct Analysis of Generated Images
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• AI Marker-based Large-scale AI Literature Mining
• Improving Stability of LS-GANs for Audio and Speech Signals
• Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System
• Extremal learning: extremizing the output of a neural network in regression problems
• An Empirical Study on GANs with Margin Cosine Loss and Relativistic Discriminator
• Enhanced Magnetic Resonance Image Synthesis with Contrast-Aware Generative Adversarial Networks