Self-Supervised GAN to Counter Forgetting

Abstract

GANs involve training two networks in an adversarial game, where each network's task depends on its adversary. Recently, several works have framed GAN training as an online or continual learning problem. We focus on the discriminator, which must perform classification under an (adversarially) shifting data distribution. When trained on sequential tasks, neural networks exhibit \emph{forgetting}. For GANs, discriminator forgetting leads to training instability. To counter forgetting, we encourage the discriminator to maintain useful representations by adding a self-supervision. Conditional GANs have a similar effect using labels. However, our self-supervised GAN does not require labels, and closes the performance gap between conditional and unconditional models. We show that, in doing so, the self-supervised discriminator learns better representations than regular GANs.

NeurIPS'18 Continual Learning workshop

References in corpus (11)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Which Training Methods for GANs do actually Converge?
• Assessing Generative Models via Precision and Recall
• Continual Learning in Generative Adversarial Nets
• On Self Modulation for Generative Adversarial Networks
• Generative Adversarial Network Training is a Continual Learning Problem
• Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks
• Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets

Cited by in corpus (3)

• Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
• An Improved Self-supervised GAN via Adversarial Training