Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game

Abstract

Self-supervised (SS) learning is a powerful approach for representation learning using unlabeled data. Recently, it has been applied to Generative Adversarial Networks (GAN) training. Specifically, SS tasks were proposed to address the catastrophic forgetting issue in the GAN discriminator. In this work, we perform an in-depth analysis to understand how SS tasks interact with learning of generator. From the analysis, we identify issues of SS tasks which allow a severely mode-collapsed generator to excel the SS tasks. To address the issues, we propose new SS tasks based on a multi-class minimax game. The competition between our proposed SS tasks in the game encourages the generator to learn the data distribution and generate diverse samples. We provide both theoretical and empirical analysis to support that our proposed SS tasks have better convergence property. We conduct experiments to incorporate our proposed SS tasks into two different GAN baseline models. Our approach establishes state-of-the-art FID scores on CIFAR-10, CIFAR-100, STL-10, CelebA, Imagenet $32\times 32$ and Stacked-MNIST datasets, outperforming existing works by considerable margins in some cases. Our unconditional GAN model approaches performance of conditional GAN without using labeled data. Our code: https://github.com/tntrung/msgan

Accepted at NeurIPS 2019

Cited by in corpus (19)

• Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis
• Data-Efficient Instance Generation from Instance Discrimination
• Exploring DeshuffleGANs in Self-Supervised Generative Adversarial Networks
• Self-Diagnosing GAN: Diagnosing Underrepresented Samples in Generative Adversarial Networks
• cGANs with Multi-Hinge Loss
• DeshuffleGAN: A Self-Supervised GAN to Improve Structure Learning
• Self-Supervised GANs with Label Augmentation
• MatchGAN: A Self-Supervised Semi-Supervised Conditional Generative Adversarial Network
• Score-Guided Generative Adversarial Networks
• Measuring Fairness in Generative Models
• Towards Self-learning Edge Intelligence in 6G
• Adaptive Weighted Discriminator for Training Generative Adversarial Networks
• SS-CPGAN: Self-Supervised Cut-and-Pasting Generative Adversarial Network for Object Segmentation
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• Noise Homogenization via Multi-Channel Wavelet Filtering for High-Fidelity Sample Generation in GANs
• Improved Image Generation via Sparse Modeling
• Backpropagating through Fréchet Inception Distance
• EXoN: EXplainable encoder Network
• Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach