A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications

Abstract

Generative adversarial networks (GANs) are a hot research topic recently. GANs have been widely studied since 2014, and a large number of algorithms have been proposed. However, there is few comprehensive study explaining the connections among different GANs variants, and how they have evolved. In this paper, we attempt to provide a review on various GANs methods from the perspectives of algorithms, theory, and applications. Firstly, the motivations, mathematical representations, and structure of most GANs algorithms are introduced in details. Furthermore, GANs have been combined with other machine learning algorithms for specific applications, such as semi-supervised learning, transfer learning, and reinforcement learning. This paper compares the commonalities and differences of these GANs methods. Secondly, theoretical issues related to GANs are investigated. Thirdly, typical applications of GANs in image processing and computer vision, natural language processing, music, speech and audio, medical field, and data science are illustrated. Finally, the future open research problems for GANs are pointed out.

References in corpus (181)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Generative Adversarial Networks: An Overview
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Categorical Reparameterization with Gumbel-Softmax
• Striving for Simplicity: The All Convolutional Net
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Synthesis With Auxiliary Classifier GANs
• GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification
• Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Improved Training of Wasserstein GANs
• Conditional Adversarial Domain Adaptation
• Improved Techniques for Training GANs
• NIPS 2016 Tutorial: Generative Adversarial Networks
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Toward Multimodal Image-to-Image Translation
• Energy-based Generative Adversarial Network
• Generating Videos with Scene Dynamics
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Adversarial Feature Learning
• Adversarially Learned Inference
• Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Invertible Conditional GANs for image editing
• The relativistic discriminator: a key element missing from standard GAN
• Unsupervised Image-to-Image Translation Networks
• Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models
• Adversarial Discriminative Domain Adaptation
• Decomposing Motion and Content for Natural Video Sequence Prediction
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• A note on the evaluation of generative models
• cGANs with Projection Discriminator
• Perceptual Adversarial Networks for Image-to-Image Transformation
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN
• Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
• Generative Adversarial Imitation Learning
• SalGAN: Visual Saliency Prediction with Generative Adversarial Networks
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Deep Generative Stochastic Networks Trainable by Backprop
• DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• Multimodal Unsupervised Image-to-Image Translation
• Wasserstein Distance Guided Representation Learning for Domain Adaptation
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs
• Semantic Image Synthesis with Spatially-Adaptive Normalization
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Neural Photo Editing with Introspective Adversarial Networks
• Domain-Adversarial Neural Networks
• Mode Regularized Generative Adversarial Networks
• Good Semi-supervised Learning that Requires a Bad GAN
• Triple Generative Adversarial Nets
• An empirical study on evaluation metrics of generative adversarial networks
• Unsupervised Learning of Disentangled Representations from Video
• Generative Image Inpainting with Contextual Attention
• GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Adversarial Learning for Neural Dialogue Generation
• Generating Steganographic Images via Adversarial Training
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Stabilizing Training of Generative Adversarial Networks through Regularization
• Continual Learning with Deep Generative Replay
• Learning a Driving Simulator
• Learning Adversarially Fair and Transferable Representations
• Learning in Implicit Generative Models
• GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
• Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Demystifying MMD GANs
• Representation Learning by Rotating Your Faces
• A Variational Inequality Perspective on Generative Adversarial Networks
• Adversarial Examples that Fool both Computer Vision and Time-Limited Humans
• CycleGAN, a Master of Steganography
• Generating Handwritten Chinese Characters using CycleGAN
• Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks
• Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery
• Adversarial Video Generation on Complex Datasets
• Emergent Complexity via Multi-Agent Competition
• AdaGAN: Boosting Generative Models
• MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks
• Synthesizing Programs for Images using Reinforced Adversarial Learning
• On the regularization of Wasserstein GANs
• Generative Adversarial Active Learning
• Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis
• Multi-Agent Generative Adversarial Imitation Learning
• Video-to-Video Synthesis
• Dual Discriminator Generative Adversarial Nets
• Learning to Protect Communications with Adversarial Neural Cryptography
• Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
• ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity?
• Texture Synthesis with Spatial Generative Adversarial Networks
• Boundary-Seeking Generative Adversarial Networks
• Towards Diverse and Natural Image Descriptions via a Conditional GAN
• Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
• Censoring Representations with an Adversary
• Generative Adversarial User Model for Reinforcement Learning Based Recommendation System
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• Connecting Generative Adversarial Networks and Actor-Critic Methods
• Perceptual Generative Adversarial Networks for Small Object Detection
• Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN
• On distinguishability criteria for estimating generative models
• Binary Generative Adversarial Networks for Image Retrieval
• Unsupervised Learning of Visual Structure using Predictive Generative Networks
• Few-Shot Unsupervised Image-to-Image Translation
• APE-GAN: Adversarial Perturbation Elimination with GAN
• Triangle Generative Adversarial Networks
• CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training
• MirrorGAN: Learning Text-to-image Generation by Redescription
• Fully Convolutional Adaptation Networks for Semantic Segmentation
• Feedback GAN (FBGAN) for DNA: a Novel Feedback-Loop Architecture for Optimizing Protein Functions
• VITAL: VIsual Tracking via Adversarial Learning
• Learning a High Fidelity Pose Invariant Model for High-resolution Face Frontalization
• One-Sided Unsupervised Domain Mapping
• MAGAN: Margin Adaptation for Generative Adversarial Networks
• Multi-Generator Generative Adversarial Nets
• RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution
• SSGAN: Secure Steganography Based on Generative Adversarial Networks
• GeneGAN: Learning Object Transfiguration and Attribute Subspace from Unpaired Data
• Learning Beyond Human Expertise with Generative Models for Dental Restorations
• Face Aging With Conditional Generative Adversarial Networks
• 3D-Aware Scene Manipulation via Inverse Graphics
• KBGAN: Adversarial Learning for Knowledge Graph Embeddings
• Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis
• FairGAN: Fairness-aware Generative Adversarial Networks
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs
• Semi-supervised Conditional GANs
• Answer-based Adversarial Training for Generating Clarification Questions
• Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks
• Generative Adversarial Self-Imitation Learning
• Semantic Image Synthesis via Adversarial Learning
• Adversarially Regularized Autoencoders
• On the "steerability" of generative adversarial networks
• Generative Adversarial Network Training is a Continual Learning Problem
• Adversarial Generation of Training Examples: Applications to Moving Vehicle License Plate Recognition
• Understanding GANs: the LQG Setting
• Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis
• Neural Face Editing with Intrinsic Image Disentangling
• Approximability of Discriminators Implies Diversity in GANs
• Semi-Latent GAN: Learning to generate and modify facial images from attributes
• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• Unsupervised Visual Attribute Transfer with Reconfigurable Generative Adversarial Networks
• Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
• How Well Generative Adversarial Networks Learn Distributions
• Arbitrary Talking Face Generation via Attentional Audio-Visual Coherence Learning
• AlphaGAN: Generative adversarial networks for natural image matting
• SCAN: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-rays
• McGan: Mean and Covariance Feature Matching GAN
• Robust Conditional Generative Adversarial Networks
• Comparative Study on Generative Adversarial Networks
• Unsupervised Cipher Cracking Using Discrete GANs
• Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently
• No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling
• HyperGAN: A Generative Model for Diverse, Performant Neural Networks
• Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
• Theoretical limitations of Encoder-Decoder GAN architectures
• SRDGAN: learning the noise prior for Super Resolution with Dual Generative Adversarial Networks
• An Online Learning Approach to Generative Adversarial Networks
• TGAN: Deep Tensor Generative Adversarial Nets for Large Image Generation
• CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation
• Image Generation and Editing with Variational Info Generative AdversarialNetworks
• Energy-Based Sequence GANs for Recommendation and Their Connection to Imitation Learning
• Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs
• Biphasic Learning of GANs for High-Resolution Image-to-Image Translation
• Synthetic-Neuroscore: Using A Neuro-AI Interface for Evaluating Generative Adversarial Networks
• Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference
• Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
• Environment Reconstruction with Hidden Confounders for Reinforcement Learning based Recommendation
• Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)
• A Layer-Based Sequential Framework for Scene Generation with GANs

Cited by in corpus (40)

• TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
• Applications of Deep Learning Techniques for Automated Multiple Sclerosis Detection Using Magnetic Resonance Imaging: A Review
• An overview of artificial intelligence techniques for diagnosis of Schizophrenia based on magnetic resonance imaging modalities: Methods, challenges, and future works
• Generative Adversarial Networks for Spatio-temporal Data: A Survey
• A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization
• A survey on GANs for computer vision: Recent research, analysis and taxonomy
• Enhanced Balancing GAN: Minority-class Image Generation
• A review on Deep Reinforcement Learning for Fluid Mechanics
• A review on deep reinforcement learning for fluid mechanics: an update
• Review: Deep Learning in Electron Microscopy
• Generative adversarial networks in time series: A survey and taxonomy
• Creativity and Machine Learning: A Survey
• Quaternion Generative Adversarial Networks
• Quantum semi-supervised generative adversarial network for enhanced data classification
• An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Training
• Generative machine learning methods for multivariate ensemble post-processing
• A Comprehensive Survey on Self-Supervised Learning for Recommendation
• Generative Partial Multi-View Clustering
• ImGAGN:Imbalanced Network Embedding via Generative Adversarial Graph Networks
• GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework
• Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
• Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?
• Combining machine learning with physics: A framework for tracking and sorting multiple dark solitons
• Double Generative Adversarial Networks for Conditional Independence Testing
• Direct May Not Be the Best: An Incremental Evolution View of Pose Generation
• Deep Learning Models in Software Requirements Engineering
• Learning Robust Algorithms for Online Allocation Problems Using Adversarial Training
• Multi-view Deep One-class Classification: A Systematic Exploration
• A Mathematical Framework for Learning Probability Distributions
• A Novel Framework for Selection of GANs for an Application
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• Exploiting the Hidden Tasks of GANs: Making Implicit Subproblems Explicit
• An Overview on Generative AI at Scale with Edge-Cloud Computing
• Differentially Private Generation of Small Images
• Testing Directed Acyclic Graph via Structural, Supervised and Generative Adversarial Learning
• Aerodynamic Data Predictions Based on Multi-task Learning
• Two-stage Training for Learning from Label Proportions
• Making Robots Draw A Vivid Portrait In Two Minutes
• On Leveraging Unlabeled Data for Concurrent Positive-Unlabeled Classification and Robust Generation
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey