A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning
arXiv:1902.02992
Abstract
Hyperbolic space is a geometry that is known to be well-suited for representation learning of data with an underlying hierarchical structure. In this paper, we present a novel hyperbolic distribution called \textit{pseudo-hyperbolic Gaussian}, a Gaussian-like distribution on hyperbolic space whose density can be evaluated analytically and differentiated with respect to the parameters. Our distribution enables the gradient-based learning of the probabilistic models on hyperbolic space that could never have been considered before. Also, we can sample from this hyperbolic probability distribution without resorting to auxiliary means like rejection sampling. As applications of our distribution, we develop a hyperbolic-analog of variational autoencoder and a method of probabilistic word embedding on hyperbolic space. We demonstrate the efficacy of our distribution on various datasets including MNIST, Atari 2600 Breakout, and WordNet.
20 pages, 12 figures
References in corpus (7)
- Distributed Representations of Words and Phrases and their Compositionality
- Auto-Encoding Variational Bayes
- Categorical Reparameterization with Gumbel-Softmax
- Variational Inference with Normalizing Flows
- Adversarial Autoencoders with Constant-Curvature Latent Manifolds
- Poincaré Wasserstein Autoencoder
- The LORACs prior for VAEs: Letting the Trees Speak for the Data
Cited by in corpus (18)
- Riemannian Continuous Normalizing Flows
- Poincaré Wasserstein Autoencoder
- Neural Ordinary Differential Equations on Manifolds
- Latent Variable Modelling with Hyperbolic Normalizing Flows
- Variational Autoencoders with Riemannian Brownian Motion Priors
- Variational Auto-Encoder: not all failures are equal
- Increasing Expressivity of a Hyperspherical VAE
- Hyperbolic Deep Learning for Foundation Models: A Survey
- Variational Autoencoder with Learned Latent Structure
- Hyperbolic Generative Adversarial Network
- Semi-Supervised Hierarchical Drug Embedding in Hyperbolic Space
- Hyperbolic Neural Networks++
- Amortised Learning by Wake-Sleep
- Equivariant Manifold Flows
- CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data
- Continuous normalizing flows on manifolds
- Siamese networks for Poincaré embeddings and the reconstruction of evolutionary trees
- Understanding in Artificial Intelligence