Generative Model without Prior Distribution Matching

Abstract

Variational Autoencoder (VAE) and its variations are classic generative models by learning a low-dimensional latent representation to satisfy some prior distribution (e.g., Gaussian distribution). Their advantages over GAN are that they can simultaneously generate high dimensional data and learn latent representations to reconstruct the inputs. However, it has been observed that a trade-off exists between reconstruction and generation since matching prior distribution may destroy the geometric structure of data manifold. To mitigate this problem, we propose to let the prior match the embedding distribution rather than imposing the latent variables to fit the prior. The embedding distribution is trained using a simple regularized autoencoder architecture which preserves the geometric structure to the maximum. Then an adversarial strategy is employed to achieve a latent mapping. We provide both theoretical and experimental support for the effectiveness of our method, which alleviates the contradiction between topological properties' preserving of data manifold and distribution matching in latent space.

8 pages, 8 figures

References in corpus (11)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Neural Discrete Representation Learning
• Self-Attention Generative Adversarial Networks
• IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• Hyperspherical Variational Auto-Encoders
• Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
• Mimicry: Towards the Reproducibility of GAN Research
• Latent Variables on Spheres for Autoencoders in High Dimensions