Multiclass non-Adversarial Image Synthesis, with Application to Classification from Very Small Sample

Abstract

The generation of synthetic images is currently being dominated by Generative Adversarial Networks (GANs). Despite their outstanding success in generating realistic looking images, they still suffer from major drawbacks, including an unstable and highly sensitive training procedure, mode-collapse and mode-mixture, and dependency on large training sets. In this work we present a novel non-adversarial generative method - Clustered Optimization of LAtent space (COLA), which overcomes some of the limitations of GANs, and outperforms GANs when training data is scarce. In the full data regime, our method is capable of generating diverse multi-class images with no supervision, surpassing previous non-adversarial methods in terms of image quality and diversity. In the small-data regime, where only a small sample of labeled images is available for training with no access to additional unlabeled data, our results surpass state-of-the-art GAN models trained on the same amount of data. Finally, when utilizing our model to augment small datasets, we surpass the state-of-the-art performance in small-sample classification tasks on challenging datasets, including CIFAR-10, CIFAR-100, STL-10 and Tiny-ImageNet. A theoretical analysis supporting the essence of the method is presented.

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Survey on Deep Learning in Medical Image Analysis
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Wide Residual Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Adversarial Feature Learning
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• cGANs with Projection Discriminator
• Mode Regularized Generative Adversarial Networks
• On Convergence and Stability of GANs
• A Comprehensive Survey on Transfer Learning
• An empirical study on evaluation metrics of generative adversarial networks
• A Note on the Inception Score
• Optimizing the Latent Space of Generative Networks
• Data-dependent Initializations of Convolutional Neural Networks
• Disconnected Manifold Learning for Generative Adversarial Networks
• Gaussian Mixture Generative Adversarial Networks for Diverse Datasets, and the Unsupervised Clustering of Images
• An Improved Evaluation Framework for Generative Adversarial Networks
• Mimicry: Towards the Reproducibility of GAN Research
• A Close Look at Deep Learning with Small Data