IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis

Abstract

We present a novel introspective variational autoencoder (IntroVAE) model for synthesizing high-resolution photographic images. IntroVAE is capable of self-evaluating the quality of its generated samples and improving itself accordingly. Its inference and generator models are jointly trained in an introspective way. On one hand, the generator is required to reconstruct the input images from the noisy outputs of the inference model as normal VAEs. On the other hand, the inference model is encouraged to classify between the generated and real samples while the generator tries to fool it as GANs. These two famous generative frameworks are integrated in a simple yet efficient single-stream architecture that can be trained in a single stage. IntroVAE preserves the advantages of VAEs, such as stable training and nice latent manifold. Unlike most other hybrid models of VAEs and GANs, IntroVAE requires no extra discriminators, because the inference model itself serves as a discriminator to distinguish between the generated and real samples. Experiments demonstrate that our method produces high-resolution photo-realistic images (e.g., CELEBA images at \(1024^{2}\)), which are comparable to or better than the state-of-the-art GANs.

Cited by in corpus (53)

• Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review
• adVAE: A self-adversarial variational autoencoder with Gaussian anomaly prior knowledge for anomaly detection
• From Variational to Deterministic Autoencoders
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• SoftAdapt: Techniques for Adaptive Loss Weighting of Neural Networks with Multi-Part Loss Functions
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• PyramidBox++: High Performance Detector for Finding Tiny Face
• AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation
• A Deep Learning Generative Model Approach for Image Synthesis of Plant Leaves
• Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction
• O-GAN: Extremely Concise Approach for Auto-Encoding Generative Adversarial Networks
• Image Generation From Small Datasets via Batch Statistics Adaptation
• Image Morphing with Perceptual Constraints and STN Alignment
• COCO-GAN: Generation by Parts via Conditional Coordinating
• High Fidelity Face Manipulation with Extreme Poses and Expressions
• Dual Variational Generation for Low-Shot Heterogeneous Face Recognition
• Fake face detection via adaptive manipulation traces extraction network
• NVAE-GAN Based Approach for Unsupervised Time Series Anomaly Detection
• Variational Autoencoders for Jet Simulation
• Diffusion-Weighted Magnetic Resonance Brain Images Generation with Generative Adversarial Networks and Variational Autoencoders: A Comparison Study
• A Survey of Deep Facial Attribute Analysis
• Generalized Zero-Shot Learning via VAE-Conditioned Generative Flow
• Deep Generative Learning via Variational Gradient Flow
• Global and Local Consistent Wavelet-domain Age Synthesis
• Learning Neural Light Transport
• Dual Contradistinctive Generative Autoencoder
• UVA: A Universal Variational Framework for Continuous Age Analysis
• Fully Unsupervised Diversity Denoising with Convolutional Variational Autoencoders
• Multi-modal data generation with a deep metric variational autoencoder
• SCNet: A Generalized Attention-based Model for Crack Fault Segmentation
• Latent Space Analysis of VAE and Intro-VAE applied to 3-dimensional MR Brain Volumes of Multiple Sclerosis, Leukoencephalopathy, and Healthy Patients
• On the Latent Holes of VAEs for Text Generation
• NWT: Towards natural audio-to-video generation with representation learning
• Scalable Fine-grained Generated Image Classification Based on Deep Metric Learning
• Introducing a new high-resolution handwritten digits data set with writer characteristics
• Adversarial Attack Type I: Cheat Classifiers by Significant Changes
• Sampling Using Neural Networks for colorizing the grayscale images
• Learning geometry-image representation for 3D point cloud generation
• Interpreting Spatially Infinite Generative Models
• Bridging the ELBO and MMD
• Generative Model without Prior Distribution Matching
• VCE: Variational Convertor-Encoder for One-Shot Generalization
• Learning Interpretable and Discrete Representations with Adversarial Training for Unsupervised Text Classification
• Deep Generative Models: Deterministic Prediction with an Application in Inverse Rendering
• Wavelets to the Rescue: Improving Sample Quality of Latent Variable Deep Generative Models
• An Interpretable Generative Model for Handwritten Digit Image Synthesis
• Vehicle Image Generation Going Well with The Surroundings
• Deep Generative Variational Autoencoding for Replay Spoof Detection in Automatic Speaker Verification
• IGAN: Inferent and Generative Adversarial Networks
• A Generative Learning Approach for Spatio-temporal Modeling in Connected Vehicular Network
• Deep Automodulators
• Learning to Conceal: A Deep Learning Based Method for Preserving Privacy and Avoiding Prejudice
• Variational Hetero-Encoder Randomized GANs for Joint Image-Text Modeling