Self-Supervised GANs with Label Augmentation

Abstract

Recently, transformation-based self-supervised learning has been applied to generative adversarial networks (GANs) to mitigate catastrophic forgetting in the discriminator by introducing a stationary learning environment. However, the separate self-supervised tasks in existing self-supervised GANs cause a goal inconsistent with generative modeling due to the fact that their self-supervised classifiers are agnostic to the generator distribution. To address this problem, we propose a novel self-supervised GAN that unifies the GAN task with the self-supervised task by augmenting the GAN labels (real or fake) via self-supervision of data transformation. Specifically, the original discriminator and self-supervised classifier are unified into a label-augmented discriminator that predicts the augmented labels to be aware of both the generator distribution and the data distribution under every transformation, and then provide the discrepancy between them to optimize the generator. Theoretically, we prove that the optimal generator could converge to replicate the real data distribution. Empirically, we show that the proposed method significantly outperforms previous self-supervised and data augmentation GANs on both generative modeling and representation learning across benchmark datasets.

arXiv commentAccepted at NeurIPS 2021

References in corpus (29)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Unsupervised Representation Learning by Predicting Image Rotations
• Improved Training of Wasserstein GANs
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Unsupervised Visual Representation Learning by Context Prediction
• On the regularization of Wasserstein GANs
• Dual Discriminator Generative Adversarial Nets
• Image Augmentations for GAN Training
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• On Self Modulation for Generative Adversarial Networks
• Stabilizing GAN Training with Multiple Random Projections
• Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game
• Lipschitz Generative Adversarial Nets
• BourGAN: Generative Networks with Metric Embeddings
• Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis
• Feature Quantization Improves GAN Training
• Improved Consistency Regularization for GANs
• Multi-objective training of Generative Adversarial Networks with multiple discriminators
• Adversarial Lipschitz Regularization
• Real or Not Real, that is the Question
• An Improved Self-supervised GAN via Adversarial Training
• GAN Memory with No Forgetting
• Slimmable Generative Adversarial Networks