Generative Adversarial Networks: An Overview

Abstract

Generative adversarial networks (GANs) provide a way to learn deep representations without extensively annotated training data. They achieve this through deriving backpropagation signals through a competitive process involving a pair of networks. The representations that can be learned by GANs may be used in a variety of applications, including image synthesis, semantic image editing, style transfer, image super-resolution and classification. The aim of this review paper is to provide an overview of GANs for the signal processing community, drawing on familiar analogies and concepts where possible. In addition to identifying different methods for training and constructing GANs, we also point to remaining challenges in their theory and application.

Accepted in the IEEE Signal Processing Magazine Special Issue on Deep Learning for Visual Understanding

References in corpus (29)

• Efficient Estimation of Word Representations in Vector Space
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Improved Techniques for Training GANs
• NIPS 2016 Tutorial: Generative Adversarial Networks
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Energy-based Generative Adversarial Network
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• A note on the evaluation of generative models
• Unrolled Generative Adversarial Networks
• Neural Photo Editing with Introspective Adversarial Networks
• Learning What and Where to Draw
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• The Numerics of GANs
• Amortised MAP Inference for Image Super-resolution
• AdaGAN: Boosting Generative Models
• Generative Visual Manipulation on the Natural Image Manifold
• On distinguishability criteria for estimating generative models
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• Generative Adversarial Nets from a Density Ratio Estimation Perspective
• Stacked Generative Adversarial Networks
• Inverting The Generator Of A Generative Adversarial Network (II)
• DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data
• Inverting The Generator Of A Generative Adversarial Network

Cited by in corpus (220)

• Deep Learning for Anomaly Detection: A Review
• An overview of deep learning in medical imaging focusing on MRI
• Generative Adversarial Network in Medical Imaging: A Review
• Normalizing Flows: An Introduction and Review of Current Methods
• A trans-disciplinary review of deep learning research for water resources scientists
• Quantum generative adversarial networks
• Efficient GAN-Based Anomaly Detection
• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• Generating 3D structures from a 2D slice with GAN-based dimensionality expansion
• How to Build a Graph-Based Deep Learning Architecture in Traffic Domain: A Survey
• BS-Nets: An End-to-End Framework For Band Selection of Hyperspectral Image
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Revolutionizing Future Connectivity: A Contemporary Survey on AI-empowered Satellite-based Non-Terrestrial Networks in 6G
• Machine-Learning-Assisted Metasurface Design for High-Efficiency Thermal Emitter Optimization
• GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
• Adversarial Text-to-Image Synthesis: A Review
• Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold
• Deep learning-based synthetic-CT generation in radiotherapy and PET: a review
• Artificial Intelligence for Satellite Communication: A Review
• A Survey on Deep Learning for Skin Lesion Segmentation
• On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques
• An overview of artificial intelligence techniques for diagnosis of Schizophrenia based on magnetic resonance imaging modalities: Methods, challenges, and future works
• Image Colorization with Generative Adversarial Networks
• Quantum generative adversarial learning in a superconducting quantum circuit
• Generative AI Literacy: Twelve Defining Competencies
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Generative Adversarial Networks (GANs) in Networking: A Comprehensive Survey & Evaluation
• Machine learning in nuclear materials research
• Automatic autism spectrum disorder detection using artificial intelligence methods with MRI neuroimaging: A review
• Automated Detection and Forecasting of COVID-19 using Deep Learning Techniques: A Review
• Data-driven modelling of nonlinear spatio-temporal fluid flows using a deep convolutional generative adversarial network
• Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges
• Explainable Machine Learning using Real, Synthetic and Augmented Fire Tests to Predict Fire Resistance and Spalling of RC Columns
• Understanding Event-Generation Networks via Uncertainties
• Towards quantum enhanced adversarial robustness in machine learning
• AirSLAM: An Efficient and Illumination-Robust Point-Line Visual SLAM System
• A review of Generative Adversarial Networks for Electronic Health Records: applications, evaluation measures and data sources
• Bi-Modality Medical Image Synthesis Using Semi-Supervised Sequential Generative Adversarial Networks
• Gradient-based deterministic inversion of geophysical data with Generative Adversarial Networks: is it feasible?
• GANplifying Event Samples
• Generative Networks for Precision Enthusiasts
• cellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM
• Unsupervised Deep Learning for IoT Time Series
• Medical image registration using unsupervised deep neural network: A scoping literature review
• Anomaly Detection in Industrial Machinery using IoT Devices and Machine Learning: a Systematic Mapping
• Stabilizing Generative Adversarial Networks: A Survey
• Semantic Invariant Multi-view Clustering with Fully Incomplete Information
• Deep Learning for Environmentally Robust Speech Recognition: An Overview of Recent Developments
• DC-AL GAN: Pseudoprogression and True Tumor Progression of Glioblastoma Multiform Image Classification Based on DCGAN and AlexNet
• A Physics-Guided Modular Deep-Learning Based Automated Framework for Tumor Segmentation in PET Images
• De novo Design of Polymer Electrolytes with High Conductivity using GPT-based and Diffusion-based Generative Models
• A Spatiotemporal Deep Neural Network for Fine-Grained Multi-Horizon Wind Prediction
• TCGAN: Convolutional Generative Adversarial Network for Time Series Classification and Clustering
• Graph Pooling via Coarsened Graph Infomax
• Classical and learned MR to pseudo-CT mappings for accurate transcranial ultrasound simulation
• Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey
• Outlier-based Autism Detection using Longitudinal Structural MRI
• Presenting Unbinned Differential Cross Section Results
• DCTRGAN: Improving the Precision of Generative Models with Reweighting
• Artificial intelligence and machine learning applications for cultured meat
• Uncertainty-aware Surrogate Models for Airfoil Flow Simulations with Denoising Diffusion Probabilistic Models
• Generating Synthetic Mobility Networks with Generative Adversarial Networks
• Deep Learning with Cinematic Rendering: Fine-Tuning Deep Neural Networks Using Photorealistic Medical Images
• RadioGAT: A Joint Model-based and Data-driven Framework for Multi-band Radiomap Reconstruction via Graph Attention Networks
• SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
• Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives
• Generative Adversarial Networks (GAN) for compact beam source modelling in Monte Carlo simulations
• A Comprehensive Survey of Machine Learning Applied to Radar Signal Processing
• Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
• Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments
• Machine Learning-Driven Process of Alumina Ceramics Laser Machining
• Towards a Deep Learning Model for Hadronization
• Deep neural networks for fast acquisition of aortic 3D pressure and velocity flow fields
• Unsupervised Learning of Anomaly Detection from Contaminated Image Data using Simultaneous Encoder Training
• Synthesis of Realistic ECG using Generative Adversarial Networks
• DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series
• A performance characterization of quantum generative models
• HashVFL: Defending Against Data Reconstruction Attacks in Vertical Federated Learning
• Extracting particle size distribution from laser speckle with a physics-enhanced autocorrelation-based estimator (PEACE)
• Accelerated and Quantitative 3D Semisolid MT/CEST Imaging using a Generative Adversarial Network (GAN-CEST)
• Reviewing Intelligent Cinematography: AI research for camera-based video production
• SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks
• Noise robustness and experimental demonstration of a quantum generative adversarial network for continuous distributions
• Generative Adversarial Networks for Financial Trading Strategies Fine-Tuning and Combination
• Anomaly Detection under Coordinate Transformations
• Adversarial Semantic Scene Completion from a Single Depth Image
• Adversarially Learned Anomaly Detection
• CosSIF: Cosine similarity-based image filtering to overcome low inter-class variation in synthetic medical image datasets
• Deep Learning for Optical Tweezers
• MillimeterDL: Deep Learning Simulations of the Microwave Sky
• Generative Multi-Form Bayesian Optimization
• Adversarial Learning for Improved Onsets and Frames Music Transcription
• AutoScore-Imbalance: An interpretable machine learning tool for development of clinical scores with rare events data
• Defend Data Poisoning Attacks on Voice Authentication
• Towards Robust Stability Prediction in Smart Grids: GAN-based Approach under Data Constraints and Adversarial Challenges
• ELSA -- Enhanced latent spaces for improved collider simulations
• Relaxed Wasserstein with Applications to GANs
• Experimental Investigation of Machine Learning based Soft-Failure Management using the Optical Spectrum
• Generative Adversarial Network for Handwritten Text
• Procedural Content Generation using Neuroevolution and Novelty Search for Diverse Video Game Levels
• Least $k$th-Order and Rényi Generative Adversarial Networks
• Exploring Long- and Short-Range Temporal Information for Learned Video Compression
• Designing thermal radiation metamaterials via hybrid adversarial autoencoder and Bayesian optimization
• The Six Fronts of the Generative Adversarial Networks
• Synthetic Data Generation for Fraud Detection using GANs
• Content-Based Collaborative Generation for Recommender Systems
• Sdf-GAN: Semi-supervised Depth Fusion with Multi-scale Adversarial Networks
• CGAN-EB: A Non-parametric Empirical Bayes Method for Crash Hotspot Identification Using Conditional Generative Adversarial Networks: A Simulated Crash Data Study
• Data-driven geophysics: from dictionary learning to deep learning
• Text to Image Synthesis using Stacked Conditional Variational Autoencoders and Conditional Generative Adversarial Networks
• A Style-Aware Content Loss for Real-time HD Style Transfer
• Generative Aging of Brain Images with Diffeomorphic Registration
• Deep learning in automated ultrasonic NDE -- developments, axioms and opportunities
• Neural Network Methods for Radiation Detectors and Imaging
• Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices
• Generative ensemble deep learning severe weather prediction from a deterministic convection-allowing model
• PolyGAN: High-Order Polynomial Generators
• Emerging Directions in Geophysical Inversion
• Image Projective Transformation Rectification with Synthetic Data for Smartphone-captured Chest X-ray Photos Classification
• A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis
• Multi-Mapping Image-to-Image Translation with Central Biasing Normalization
• Optimal Data Generation in Multi-Dimensional Parameter Spaces, using Bayesian Optimization
• Active inference and deep generative modeling for cognitive ultrasound
• Editing in Style: Uncovering the Local Semantics of GANs
• Recent Advances of Image Steganography with Generative Adversarial Networks
• Structural Forecasting for Short-term Tropical Cyclone Intensity Guidance
• A tomographic spherical mass map emulator of the KiDS-1000 survey using conditional generative adversarial networks
• Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients
• Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications
• Recovery from Adversarial Attacks in Cyber-physical Systems: Shallow, Deep and Exploratory Works
• MammoGANesis: Controlled Generation of High-Resolution Mammograms for Radiology Education
• Enhancing Clinical Documentation with Synthetic Data: Leveraging Generative Models for Improved Accuracy
• Mixture of Experts for Decentralized Generative AI and Reinforcement Learning in Wireless Networks: A Comprehensive Survey
• Self-Supervised Learning for Text Recognition: A Critical Survey
• SeisFusion: Constrained Diffusion Model with Input Guidance for 3D Seismic Data Interpolation and Reconstruction
• Reviewing FID and SID Metrics on Generative Adversarial Networks
• A Survey on Deep Learning for Neuroimaging-based Brain Disorder Analysis
• MFCCGAN: A Novel MFCC-Based Speech Synthesizer Using Adversarial Learning
• On Attribution of Deepfakes
• AI Simulation by Digital Twins: Systematic Survey, Reference Framework, and Mapping to a Standardized Architecture
• Stimulate the Potential of Robots via Competition
• Data-Driven Modeling of Noise Time Series with Convolutional Generative Adversarial Networks
• Affective social anthropomorphic intelligent system
• ReCLIP++: Learn to Rectify the Bias of CLIP for Unsupervised Semantic Segmentation
• Strategic Application of AIGC for UAV Trajectory Design: A Channel Knowledge Map Approach
• How to GAN Event Unweighting
• Diffusion-Weighted Magnetic Resonance Brain Images Generation with Generative Adversarial Networks and Variational Autoencoders: A Comparison Study
• Designing GANs: A Likelihood Ratio Approach
• SeismoGen: Seismic Waveform Synthesis Using Generative Adversarial Networks
• Physical Artificial Intelligence: The Concept Expansion of Next-Generation Artificial Intelligence
• Are Deep Learning-Generated Social Media Profiles Indistinguishable from Real Profiles?
• Outage Detection in Partially Observable Distribution Systems using Smart Meters and Generative Adversarial Networks
• Feasibility of Modeling Orthogonal Frequency-Division Multiplexing Communication Signals with Unsupervised Generative Adversarial Networks
• Hierarchical Mixtures of Generators for Adversarial Learning
• Diffusion Augmented Retrieval: A Training-Free Approach to Interactive Text-to-Image Retrieval
• GRD-Net: Generative-Reconstructive-Discriminative Anomaly Detection with Region of Interest Attention Module
• Snore-GANs: Improving Automatic Snore Sound Classification with Synthesized Data
• Tarallo: Evading Behavioral Malware Detectors in the Problem Space
• SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Driving
• Score-Guided Generative Adversarial Networks
• Generative modelling for mass-mapping with fast uncertainty quantification
• Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I
• Joint Learning of Generative Translator and Classifier for Visually Similar Classes
• Generative adversarial network based on chaotic time series
• An Evaluation Study of Generative Adversarial Networks for Collaborative Filtering
• Estimation with Uncertainty via Conditional Generative Adversarial Networks
• Landmarks Augmentation with Manifold-Barycentric Oversampling
• Augmentation-Interpolative AutoEncoders for Unsupervised Few-Shot Image Generation
• Improving ensemble extreme precipitation forecasts using generative artificial intelligence
• Multi-View Adaptive Contrastive Learning for Information Retrieval Based Fault Localization
• Image-Text Out-Of-Context Detection Using Synthetic Multimodal Misinformation
• Reviewing continual learning from the perspective of human-level intelligence
• Human Fall Detection using Transfer Learning-based 3D CNN
• Conditional Generative Models for Learning Stochastic Processes
• Morphomics via Next-generation Electron Microscopy
• Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey
• NIDA-CLIFGAN: Natural Infrastructure Damage Assessment through Efficient Classification Combining Contrastive Learning, Information Fusion and Generative Adversarial Networks
• AMAD: Adversarial Multiscale Anomaly Detection on High-Dimensional and Time-Evolving Categorical Data
• Fluid Grey 2: How Well Does Generative Adversarial Network Learn Deeper Topology Structure in Architecture That Matches Images?
• Latent Space Refinement for Deep Generative Models
• Speech2Video: Cross-Modal Distillation for Speech to Video Generation
• Machine Learning for Naval Architecture, Ocean and Marine Engineering
• Manifold-preserved GANs
• Protecting Intellectual Property of Generative Adversarial Networks from Ambiguity Attack
• Real-time Burst Photo Selection Using a Light-Head Adversarial Network
• R3Net: Random Weights, Rectifier Linear Units and Robustness for Artificial Neural Network
• Shadow Feature Refinement Network: Progressive Feature Refinement based on Knowledge Distillation for Effective Shadow Removal
• ReCon: Reconfiguring Analog Rydberg Atom Quantum Computers for Quantum Generative Adversarial Networks
• Ano-Graph: Learning Normal Scene Contextual Graphs to Detect Video Anomalies
• CoPE: Conditional image generation using Polynomial Expansions
• Practical Attacks on Voice Spoofing Countermeasures
• Unsupervised Industrial Anomaly Detection via Pattern Generative and Contrastive Networks
• Voice command generation using Progressive Wavegans
• State-observation augmented diffusion model for nonlinear assimilation with unknown dynamics
• MPC-MPNet: Model-Predictive Motion Planning Networks for Fast, Near-Optimal Planning under Kinodynamic Constraints
• What augmentations are sensitive to hyper-parameters and why?
• Spoof Face Detection Via Semi-Supervised Adversarial Training
• Learning Gaze-aware Compositional GAN
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• An Overview on Generative AI at Scale with Edge-Cloud Computing
• Generative networks as inverse problems with fractional wavelet scattering networks
• Ocular Disease Classification Using CNN with Deep Convolutional Generative Adversarial Network
• Quantum data generation in a denoising model with multiscale entanglement renormalization network
• A Deep-Learning Framework for Land-Sliding Classification from Remote Sensing Image
• Generative network complex (GNC) for drug discovery
• Shape Constrained Network for Eye Segmentation in the Wild
• D-LORD for Motion Stylization
• Collaborating Foundation Models for Domain Generalized Semantic Segmentation
• A Content Transformation Block For Image Style Transfer
• Generating Long-term Continuous Multi-type Generation Profiles
• Semantic Preserving Generative Adversarial Models
• Embedding-reparameterization procedure for manifold-valued latent variables in generative models
• Adversarial Machine Learning in Text Analysis and Generation
• ReshapeGAN: Object Reshaping by Providing A Single Reference Image
• Extremal learning: extremizing the output of a neural network in regression problems
• EXACT: How to Train Your Accuracy
• Recursive InPainting (RIP): how much information is lost under recursive inferences?
• Representation learning with CGAN for casual inference
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey