Improving Variational Autoencoder with Deep Feature Consistent and Generative Adversarial Training

Abstract

We present a new method for improving the performances of variational autoencoder (VAE). In addition to enforcing the deep feature consistent principle thus ensuring the VAE output and its corresponding input images to have similar deep features, we also implement a generative adversarial training mechanism to force the VAE to output realistic and natural images. We present experimental results to show that the VAE trained with our new method outperforms state of the art in generating face images with much clearer and more natural noses, eyes, teeth, hair textures as well as reasonable backgrounds. We also show that our method can learn powerful embeddings of input face images, which can be used to achieve facial attribute manipulation. Moreover we propose a multi-view feature extraction strategy to extract effective image representations, which can be used to achieve state of the art performance in facial attribute prediction.

Accepted in Neurocomputing, 2019. arXiv admin note: text overlap with arXiv:1610.00291

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distributed Representations of Words and Phrases and their Compositionality
• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Fully Convolutional Networks for Semantic Segmentation
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Semi-Supervised Learning with Deep Generative Models
• Understanding Neural Networks Through Deep Visualization
• Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
• Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
• On Unifying Deep Generative Models

Cited by in corpus (5)

• A multimodal dynamical variational autoencoder for audiovisual speech representation learning
• GuidedStyle: Attribute Knowledge Guided Style Manipulation for Semantic Face Editing
• Achieving Explainability for Plant Disease Classification with Disentangled Variational Autoencoders
• NWT: Towards natural audio-to-video generation with representation learning
• Energy-Based Prior Latent Space Diffusion model for Reconstruction of Lumbar Vertebrae from Thick Slice MRI