LaDDer: Latent Data Distribution Modelling with a Generative Prior
arXiv:2009.00088
Abstract
In this paper, we show that the performance of a learnt generative model is closely related to the model's ability to accurately represent the inferred \textbf{latent data distribution}, i.e. its topology and structural properties. We propose LaDDer to achieve accurate modelling of the latent data distribution in a variational autoencoder framework and to facilitate better representation learning. The central idea of LaDDer is a meta-embedding concept, which uses multiple VAE models to learn an embedding of the embeddings, forming a ladder of encodings. We use a non-parametric mixture as the hyper prior for the innermost VAE and learn all the parameters in a unified variational framework. From extensive experiments, we show that our LaDDer model is able to accurately estimate complex latent distribution and results in improvement in the representation quality. We also propose a novel latent space interpolation method that utilises the derived data distribution.
Accepted to BMVC 2020. Code and demos are available at: https://github.com/lin-shuyu/ladder-latent-data-distribution-modelling
References in corpus (7)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Composing graphical models with neural networks for structured representations and fast inference
- Improving Variational Inference with Inverse Autoregressive Flow
- Improving Variational Auto-Encoders using Householder Flow
- Nonparametric Variational Auto-encoders for Hierarchical Representation Learning
- Balancing Reconstruction Quality and Regularisation in ELBO for VAEs