Self-Supervised GANs via Auxiliary Rotation Loss

Abstract

Conditional GANs are at the forefront of natural image synthesis. The main drawback of such models is the necessity for labeled data. In this work we exploit two popular unsupervised learning techniques, adversarial training and self-supervision, and take a step towards bridging the gap between conditional and unconditional GANs. In particular, we allow the networks to collaborate on the task of representation learning, while being adversarial with respect to the classic GAN game. The role of self-supervision is to encourage the discriminator to learn meaningful feature representations which are not forgotten during training. We test empirically both the quality of the learned image representations, and the quality of the synthesized images. Under the same conditions, the self-supervised GAN attains a similar performance to state-of-the-art conditional counterparts. Finally, we show that this approach to fully unsupervised learning can be scaled to attain an FID of 23.4 on unconditional ImageNet generation.

References in corpus (19)

• Overcoming catastrophic forgetting in neural networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Continual Learning Through Synaptic Intelligence
• Which Training Methods for GANs do actually Converge?
• Modulating early visual processing by language
• A Note on the Inception Score
• Assessing Generative Models via Precision and Recall
• Improvements to context based self-supervised learning
• Continual Learning in Generative Adversarial Nets
• Grasp2Vec: Learning Object Representations from Self-Supervised Grasping
• On Self Modulation for Generative Adversarial Networks
• Generative Adversarial Network Training is a Continual Learning Problem
• Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks
• On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks

Cited by in corpus (2)

• Training Generative Adversarial Networks with Limited Data
• Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis