Variational Autoencoders with Riemannian Brownian Motion Priors
arXiv:2002.05227
Abstract
Variational Autoencoders (VAEs) represent the given data in a low-dimensional latent space, which is generally assumed to be Euclidean. This assumption naturally leads to the common choice of a standard Gaussian prior over continuous latent variables. Recent work has, however, shown that this prior has a detrimental effect on model capacity, leading to subpar performance. We propose that the Euclidean assumption lies at the heart of this failure mode. To counter this, we assume a Riemannian structure over the latent space, which constitutes a more principled geometric view of the latent codes, and replace the standard Gaussian prior with a Riemannian Brownian motion prior. We propose an efficient inference scheme that does not rely on the unknown normalizing factor of this prior. Finally, we demonstrate that this prior significantly increases model capacity using only one additional scalar parameter.
arXiv commentPublished in ICML 2020
References in corpus (18)
- Auto-Encoding Variational Bayes
- Neural Discrete Representation Learning
- Hyperspherical Variational Auto-Encoders
- VAE with a VampPrior
- Explorations in Homeomorphic Variational Auto-Encoding
- A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning
- Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders
- Distribution Matching in Variational Inference
- Stick-Breaking Variational Autoencoders
- Spherical Latent Spaces for Stable Variational Autoencoders
- Metrics for Probabilistic Geometries
- Diffusion Variational Autoencoders
- Fast and Robust Shortest Paths on Manifolds Learned from Data
- Only Bayes should learn a manifold (on the estimation of differential geometric structure from data)
- Reliable training and estimation of variance networks
- Resampled Priors for Variational Autoencoders
- Expected path length on random manifolds
- Variational Diffusion Autoencoders with Random Walk Sampling