Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art

Abstract

Since their inception in 2014, Generative Adversarial Networks (GANs) have rapidly emerged as powerful tools for generating realistic and diverse data across various domains, including computer vision and other applied areas. Consisting of a discriminative network and a generative network engaged in a Minimax game, GANs have revolutionized the field of generative modeling. In February 2018, GAN secured the leading spot on the ``Top Ten Global Breakthrough Technologies List'' issued by the Massachusetts Science and Technology Review. Over the years, numerous advancements have been proposed, leading to a rich array of GAN variants, such as conditional GAN, Wasserstein GAN, CycleGAN, and StyleGAN, among many others. This survey aims to provide a general overview of GANs, summarizing the latent architecture, validation metrics, and application areas of the most widely recognized variants. We also delve into recent theoretical developments, exploring the profound connection between the adversarial principle underlying GAN and Jensen-Shannon divergence, while discussing the optimality characteristics of the GAN framework. The efficiency of GAN variants and their model architectures will be evaluated along with training obstacles as well as training solutions. In addition, a detailed discussion will be provided, examining the integration of GANs with newly developed deep learning frameworks such as Transformers, Physics-Informed Neural Networks, Large Language models, and Diffusion models. Finally, we reveal several issues as well as future research outlines in this field.

References in corpus (65)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Conditional Generative Adversarial Nets
• Explaining and Harnessing Adversarial Examples
• Denoising Diffusion Probabilistic Models
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Intriguing properties of neural networks
• WaveNet: A Generative Model for Raw Audio
• Spectral Normalization for Generative Adversarial Networks
• Towards A Rigorous Science of Interpretable Machine Learning
• Ensemble Adversarial Training: Attacks and Defenses
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Generative Adversarial Text to Image Synthesis
• Diffusion Models Beat GANs on Image Synthesis
• Diffusion Models in Vision: A Survey
• MesoNet: a Compact Facial Video Forgery Detection Network
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Unsupervised Data Augmentation for Consistency Training
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Zero-Shot Text-to-Image Generation
• Generative Modeling by Estimating Gradients of the Data Distribution
• Training Generative Adversarial Networks with Limited Data
• k-Nearest Neighbour Classifiers: 2nd Edition (with Python examples)
• Deep Recurrent Q-Learning for Partially Observable MDPs
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Reconstruction of three-dimensional porous media using generative adversarial neural networks
• FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting
• Do ImageNet Classifiers Generalize to ImageNet?
• Universal Style Transfer via Feature Transforms
• MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
• LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Generative Adversarial Network for Medical Images (MI-GAN)
• BAGAN: Data Augmentation with Balancing GAN
• Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection
• LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
• AdaGAN: Boosting Generative Models
• Do GANs actually learn the distribution? An empirical study
• Experimental quantum adversarial learning with programmable superconducting qubits
• A survey on GANs for computer vision: Recent research, analysis and taxonomy
• A very preliminary analysis of DALL-E 2
• PPGAN: Privacy-preserving Generative Adversarial Network
• Turbulent drag reduction by anisotropic permeable substrates $-$ analysis and direct numerical simulations
• Conditional GAN for timeseries generation
• TensorLayer: A Versatile Library for Efficient Deep Learning Development
• Approximation and Convergence Properties of Generative Adversarial Learning
• W-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting
• Knowledge-based Deep Learning for Modeling Chaotic Systems
• A Test of Relative Similarity For Model Selection in Generative Models
• PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics
• Epicasting: An Ensemble Wavelet Neural Network (EWNet) for Forecasting Epidemics
• A Survey on Uncertainty Quantification Methods for Deep Learning
• MetroGAN: Simulating Urban Morphology with Generative Adversarial Network
• Review of end-to-end speech synthesis technology based on deep learning
• Towards Imperceptible and Robust Adversarial Example Attacks against Neural Networks
• Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation
• Some Theoretical Insights into Wasserstein GANs
• Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias
• Prediction of Transportation Index for Urban Patterns in Small and Medium-sized Indian Cities using Hybrid RidgeGAN Model
• UQGAN: A Unified Model for Uncertainty Quantification of Deep Classifiers trained via Conditional GANs
• Spatial sensitivity analysis for urban land use prediction with physics-constrained conditional generative adversarial networks
• DEQGAN: Learning the Loss Function for PINNs with Generative Adversarial Networks
• On the Privacy Properties of GAN-generated Samples
• PAC-Bayesian Generalization Bounds for Adversarial Generative Models
• Measure-conditional Discriminator with Stationary Optimum for GANs and Statistical Distance Surrogates

Cited by in corpus (2)

• When Geoscience Meets Generative AI and Large Language Models: Foundations, Trends, and Future Challenges
• Skew-Probabilistic Neural Networks for Learning from Imbalanced Data