DeshuffleGAN: A Self-Supervised GAN to Improve Structure Learning

Abstract

Generative Adversarial Networks (GANs) triggered an increased interest in problem of image generation due to their improved output image quality and versatility for expansion towards new methods. Numerous GAN-based works attempt to improve generation by architectural and loss-based extensions. We argue that one of the crucial points to improve the GAN performance in terms of realism and similarity to the original data distribution is to be able to provide the model with a capability to learn the spatial structure in data. To that end, we propose the DeshuffleGAN to enhance the learning of the discriminator and the generator, via a self-supervision approach. Specifically, we introduce a deshuffling task that solves a puzzle of randomly shuffled image tiles, which in turn helps the DeshuffleGAN learn to increase its expressive capacity for spatial structure and realistic appearance. We provide experimental evidence for the performance improvement in generated images, compared to the baseline methods, which is consistently observed over two different datasets.

arXiv commentAccepted at ICIP 2020

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Unsupervised Representation Learning by Predicting Image Rotations
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• The relativistic discriminator: a key element missing from standard GAN
• Geometric GAN
• Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game

Cited by in corpus (3)

• A Systematic Survey of Regularization and Normalization in GANs
• Exploring DeshuffleGANs in Self-Supervised Generative Adversarial Networks
• SS-CPGAN: Self-Supervised Cut-and-Pasting Generative Adversarial Network for Object Segmentation