An Improved Self-supervised GAN via Adversarial Training

Abstract

We propose to improve unconditional Generative Adversarial Networks (GAN) by training the self-supervised learning with the adversarial process. In particular, we apply self-supervised learning via the geometric transformation on input images and assign the pseudo-labels to these transformed images. (i) In addition to the GAN task, which distinguishes data (real) versus generated (fake) samples, we train the discriminator to predict the correct pseudo-labels of real transformed samples (classification task). Importantly, we find out that simultaneously training the discriminator to classify the fake class from the pseudo-classes of real samples for the classification task will improve the discriminator and subsequently lead better guides to train generator. (ii) The generator is trained by attempting to confuse the discriminator for not only the GAN task but also the classification task. For the classification task, the generator tries to confuse the discriminator recognizing the transformation of its output as one of the real transformed classes. Especially, we exploit that when the generator creates samples that result in a similar loss (via cross-entropy) as that of the real ones, the training is more stable and the generator distribution tends to match better the data distribution. When integrating our techniques into a state-of-the-art Auto-Encoder (AE) based-GAN model, they help to significantly boost the model's performance and also establish new state-of-the-art Fréchet Inception Distance (FID) scores in the literature of unconditional GAN for CIFAR-10 and STL-10 datasets.

References in corpus (17)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Which Training Methods for GANs do actually Converge?
• Adversarial Feature Learning
• Adversarially Learned Inference
• Autoencoding beyond pixels using a learned similarity metric
• Towards Principled Methods for Training Generative Adversarial Networks
• Adversarial Autoencoders
• On Convergence and Stability of GANs
• The Numerics of GANs
• On the regularization of Wasserstein GANs
• The Unusual Effectiveness of Averaging in GAN Training
• Self-Supervised GAN to Counter Forgetting
• Lipschitz Constrained GANs via Boundedness and Continuity

Cited by in corpus (3)

• Self-Supervised GANs with Label Augmentation
• PREGAN: Pose Randomization and Estimation for Weakly Paired Image Style Translation
• Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images