The relativistic discriminator: a key element missing from standard GAN

Abstract

In standard generative adversarial network (SGAN), the discriminator estimates the probability that the input data is real. The generator is trained to increase the probability that fake data is real. We argue that it should also simultaneously decrease the probability that real data is real because 1) this would account for a priori knowledge that half of the data in the mini-batch is fake, 2) this would be observed with divergence minimization, and 3) in optimal settings, SGAN would be equivalent to integral probability metric (IPM) GANs. We show that this property can be induced by using a relativistic discriminator which estimate the probability that the given real data is more realistic than a randomly sampled fake data. We also present a variant in which the discriminator estimate the probability that the given real data is more realistic than fake data, on average. We generalize both approaches to non-standard GAN loss functions and we refer to them respectively as Relativistic GANs (RGANs) and Relativistic average GANs (RaGANs). We show that IPM-based GANs are a subset of RGANs which use the identity function. Empirically, we observe that 1) RGANs and RaGANs are significantly more stable and generate higher quality data samples than their non-relativistic counterparts, 2) Standard RaGAN with gradient penalty generate data of better quality than WGAN-GP while only requiring a single discriminator update per generator update (reducing the time taken for reaching the state-of-the-art by 400%), and 3) RaGANs are able to generate plausible high resolutions images (256x256) from a very small sample (N=2011), while GAN and LSGAN cannot; these images are of significantly better quality than the ones generated by WGAN-GP and SGAN with spectral normalization.

https://github.com/AlexiaJM/RelativisticGAN

References in corpus (5)

• Spectral Normalization for Generative Adversarial Networks
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Towards Principled Methods for Training Generative Adversarial Networks
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• GANs beyond divergence minimization

Cited by in corpus (116)

• ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• EnlightenGAN: Deep Light Enhancement without Paired Supervision
• ESRGAN+ : Further Improving Enhanced Super-Resolution Generative Adversarial Network
• Towards Unsupervised Deep Image Enhancement with Generative Adversarial Network
• A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU
• A Fusion Adversarial Underwater Image Enhancement Network with a Public Test Dataset
• Structure-Preserving Image Super-Resolution
• Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic Images via GANs
• DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
• High-fidelity reconstruction of turbulent flow from spatially limited data using enhanced super-resolution generative adversarial network
• Super-resolution reconstruction of turbulent flow at various Reynolds numbers based on generative adversarial networks
• Deep Learning for Image Super-resolution: A Survey
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• A Systematic Survey of Regularization and Normalization in GANs
• Progressive Joint Low-light Enhancement and Noise Removal for Raw Images
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks
• Video Compression with CNN-based Post Processing
• Texture Memory-Augmented Deep Patch-Based Image Inpainting
• PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier
• Structure-Preserving Super Resolution with Gradient Guidance
• Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments
• Adversarial score matching and improved sampling for image generation
• Why Spectral Normalization Stabilizes GANs: Analysis and Improvements
• A Systematic Survey of Deep Learning-based Single-Image Super-Resolution
• MSG-GAN: Multi-Scale Gradients for Generative Adversarial Networks
• Coherent Semantic Attention for Image Inpainting
• Age-Oriented Face Synthesis with Conditional Discriminator Pool and Adversarial Triplet Loss
• Adversarial Infidelity Learning for Model Interpretation
• Diving Deeper into Underwater Image Enhancement: A Survey
• CycleGAN-based Non-parallel Speech Enhancement with an Adaptive Attention-in-attention Mechanism
• Predictive data-driven model based on generative adversarial network for premixed turbulence-combustion regimes
• SynFi: Automatic Synthetic Fingerprint Generation
• GRDN:Grouped Residual Dense Network for Real Image Denoising and GAN-based Real-world Noise Modeling
• UGAN: Untraceable GAN for Multi-Domain Face Translation
• End-to-End Chinese Landscape Painting Creation Using Generative Adversarial Networks
• Inspector Gadget: A Data Programming-based Labeling System for Industrial Images
• Adversarial System Variant Approximation to Quantify Process Model Generalization
• Self-Adversarial Learning with Comparative Discrimination for Text Generation
• Self-Enhanced Convolutional Network for Facial Video Hallucination
• Contextual colorization and denoising for low-light ultra high resolution sequences
• Creating High Resolution Images with a Latent Adversarial Generator
• Deep Exemplar-based Video Colorization
• PIRM Challenge on Perceptual Image Enhancement on Smartphones: Report
• Using Physics-Informed Super-Resolution Generative Adversarial Networks for Subgrid Modeling in Turbulent Reactive Flows
• Training Generative Adversarial Networks Via Turing Test
• MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution
• EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning
• DeshuffleGAN: A Self-Supervised GAN to Improve Structure Learning
• NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results
• On Loss Functions and Recurrency Training for GAN-based Speech Enhancement Systems
• Group Equivariant Generative Adversarial Networks
• Scene relighting with illumination estimation in the latent space on an encoder-decoder scheme
• MMGAN: Generative Adversarial Networks for Multi-Modal Distributions
• Unpaired Image Super-Resolution using Pseudo-Supervision
• Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation
• On Relativistic $f$-Divergences
• Tdcgan: Temporal Dilated Convolutional Generative Adversarial Network for End-to-end Speech Enhancement
• Privacy Leakage of SIFT Features via Deep Generative Model based Image Reconstruction
• Attentive Normalization for Conditional Image Generation
• A Two-stage Complex Network using Cycle-consistent Generative Adversarial Networks for Speech Enhancement
• Super-Resolution Perception for Industrial Sensor Data
• Natural and Realistic Single Image Super-Resolution with Explicit Natural Manifold Discrimination
• CatGAN: Category-aware Generative Adversarial Networks with Hierarchical Evolutionary Learning for Category Text Generation
• Learning a low dimensional manifold of real cancer tissue with PathologyGAN
• JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video
• MALCOM: Generating Malicious Comments to Attack Neural Fake News Detection Models
• Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization
• Adversarial Regression. Generative Adversarial Networks for Non-Linear Regression: Theory and Assessment
• A Self-supervised Approach for Adversarial Robustness
• MIASSR: An Approach for Medical Image Arbitrary Scale Super-Resolution
• Variational AutoEncoder for Reference based Image Super-Resolution
• Play as You Like: Timbre-enhanced Multi-modal Music Style Transfer
• Deep learning at scale for subgrid modeling in turbulent flows
• Conditional Hybrid GAN for Sequence Generation
• Unsupervised Image Super-Resolution with an Indirect Supervised Path
• Dual Contradistinctive Generative Autoencoder
• Evaluating Generative Adversarial Networks on Explicitly Parameterized Distributions
• Residual Channel Attention Generative Adversarial Network for Image Super-Resolution and Noise Reduction
• Microscopy Image Restoration using Deep Learning on W2S
• Generating Unobserved Alternatives
• A Novel Framework for Selection of GANs for an Application
• Additive Adversarial Learning for Unbiased Authentication
• A Two-stage Unsupervised Approach for Low light Image Enhancement
• UU-Nets Connecting Discriminator and Generator for Image to Image Translation
• Deep Learning for HDR Imaging: State-of-the-Art and Future Trends
• Gradient penalty from a maximum margin perspective
• SwiftSRGAN -- Rethinking Super-Resolution for Efficient and Real-time Inference
• Momentum Centering and Asynchronous Update for Adaptive Gradient Methods
• Disentangle Perceptual Learning through Online Contrastive Learning
• HypervolGAN: An efficient approach for GAN with multi-objective training function
• Improve GAN-based Neural Vocoder using Pointwise Relativistic LeastSquare GAN
• Adversarial Purification through Representation Disentanglement
• 0.71-Å resolution electron tomography enabled by deep learning aided information recovery
• One-Stage Inpainting with Bilateral Attention and Pyramid Filling Block
• Earth-Like: An education & outreach tool for exploring the diversity of planets like our own
• Dance Dance Generation: Motion Transfer for Internet Videos
• One-to-many Approach for Improving Super-Resolution
• Joint Demosaicing and Super-Resolution (JDSR): Network Design and Perceptual Optimization
• A Survey on Adversarial Image Synthesis
• Multi-scale Grouped Dense Network for VVC Intra Coding
• Multi-FAN: Multi-Spectral Mosaic Super-Resolution Via Multi-Scale Feature Aggregation Network
• Unpaired Image Enhancement with Quality-Attention Generative Adversarial Network
• Perception-oriented Single Image Super-Resolution via Dual Relativistic Average Generative Adversarial Networks
• Multi-Grid Back-Projection Networks
• Generating the support with extreme value losses
• ReflectNet -- A Generative Adversarial Method for Single Image Reflection Suppression
• Can adversarial training learn image captioning ?
• Learning Structral coherence Via Generative Adversarial Network for Single Image Super-Resolution
• Intelligent image synthesis to attack a segmentation CNN using adversarial learning
• Class Balancing GAN with a Classifier in the Loop
• Generating Diverse Realistic Laughter for Interactive Art
• From Perspective X-ray Imaging to Parallax-Robust Orthographic Stitching
• Bridging the Visual Gap: Wide-Range Image Blending