ContraGAN: Contrastive Learning for Conditional Image Generation

Abstract

Conditional image generation is the task of generating diverse images using class label information. Although many conditional Generative Adversarial Networks (GAN) have shown realistic results, such methods consider pairwise relations between the embedding of an image and the embedding of the corresponding label (data-to-class relations) as the conditioning losses. In this paper, we propose ContraGAN that considers relations between multiple image embeddings in the same batch (data-to-data relations) as well as the data-to-class relations by using a conditional contrastive loss. The discriminator of ContraGAN discriminates the authenticity of given samples and minimizes a contrastive objective to learn the relations between training images. Simultaneously, the generator tries to generate realistic images that deceive the authenticity and have a low contrastive loss. The experimental results show that ContraGAN outperforms state-of-the-art-models by 7.3% and 7.7% on Tiny ImageNet and ImageNet datasets, respectively. Besides, we experimentally demonstrate that contrastive learning helps to relieve the overfitting of the discriminator. For a fair comparison, we re-implement twelve state-of-the-art GANs using the PyTorch library. The software package is available at https://github.com/POSTECH-CVLab/PyTorch-StudioGAN.

34th Conference on Neural Information Processing Systems (NeurIPS 2020)

References in corpus (23)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Conditional Generative Adversarial Nets
• A Simple Framework for Contrastive Learning of Visual Representations
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Training Generative Adversarial Networks with Limited Data
• Which Training Methods for GANs do actually Converge?
• A Learned Representation For Artistic Style
• Modulating early visual processing by language
• Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit
• On Convergence and Stability of GANs
• Deep learning and face recognition: the state of the art
• The Unusual Effectiveness of Averaging in GAN Training
• LOGAN: Latent Optimisation for Generative Adversarial Networks
• On the Limitations of First-Order Approximation in GAN Dynamics
• Improved Consistency Regularization for GANs
• Whitening and Coloring batch transform for GANs

Cited by in corpus (19)

• Self-Supervised Hypergraph Transformer for Recommender Systems
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• Generative Image Inpainting with Segmentation Confusion Adversarial Training and Contrastive Learning
• Cross-Modal Contrastive Learning for Text-to-Image Generation
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• Conditional GANs with Auxiliary Discriminative Classifier
• A Multi-attribute Controllable Generative Model for Histopathology Image Synthesis
• Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
• SCGG: A Deep Structure-Conditioned Graph Generative Model
• CCGG: A Deep Autoregressive Model for Class-Conditional Graph Generation
• Bounds all around: training energy-based models with bidirectional bounds
• Dual Contrastive Loss and Attention for GANs
• Learning Generalized Hybrid Proximity Representation for Image Recognition
• Enhancing Stability in Training Conditional Generative Adversarial Networks via Selective Data Matching
• Stein Latent Optimization for Generative Adversarial Networks
• Dual Projection Generative Adversarial Networks for Conditional Image Generation
• Class Balancing GAN with a Classifier in the Loop
• Label Geometry Aware Discriminator for Conditional Generative Networks
• Improving Differentiable Architecture Search with a Generative Model