Classify and Generate: Using Classification Latent Space Representations for Image Generations

Abstract

Utilization of classification latent space information for downstream reconstruction and generation is an intriguing and a relatively unexplored area. In general, discriminative representations are rich in class-specific features but are too sparse for reconstruction, whereas, in autoencoders the representations are dense but have limited indistinguishable class-specific features, making them less suitable for classification. In this work, we propose a discriminative modeling framework that employs manipulated supervised latent representations to reconstruct and generate new samples belonging to a given class. Unlike generative modeling approaches such as GANs and VAEs that aim to model the data manifold distribution, Representation based Generations (ReGene) directly represent the given data manifold in the classification space. Such supervised representations, under certain constraints, allow for reconstructions and controlled generations using an appropriate decoder without enforcing any prior distribution. Theoretically, given a class, we show that these representations when smartly manipulated using convex combinations retain the same class label. Furthermore, they also lead to the novel generation of visually realistic images. Extensive experiments on datasets of varying resolutions demonstrate that ReGene has higher classification accuracy than existing conditional generative models while being competitive in terms of FID.

References in corpus (13)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• A Simple Framework for Contrastive Learning of Visual Representations
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Density estimation using Real NVP
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Better Mixing via Deep Representations
• Generating Diverse High-Fidelity Images with VQ-VAE-2
• Variational Autoencoder with Arbitrary Conditioning
• Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions
• Perceptual Generative Autoencoders