Latent Variable Modelling with Hyperbolic Normalizing Flows

Abstract

The choice of approximate posterior distributions plays a central role in stochastic variational inference (SVI). One effective solution is the use of normalizing flows \cut{defined on Euclidean spaces} to construct flexible posterior distributions. However, one key limitation of existing normalizing flows is that they are restricted to the Euclidean space and are ill-equipped to model data with an underlying hierarchical structure. To address this fundamental limitation, we present the first extension of normalizing flows to hyperbolic spaces. We first elevate normalizing flows to hyperbolic spaces using coupling transforms defined on the tangent bundle, termed Tangent Coupling ($\mathcal{T}\mathcal{C}$). We further introduce Wrapped Hyperboloid Coupling ($\mathcal{W}\mathbb{H}C$), a fully invertible and learnable transformation that explicitly utilizes the geometric structure of hyperbolic spaces, allowing for expressive posteriors while being efficient to sample from. We demonstrate the efficacy of our novel normalizing flow over hyperbolic VAEs and Euclidean normalizing flows. Our approach achieves improved performance on density estimation, as well as reconstruction of real-world graph data, which exhibit a hierarchical structure. Finally, we show that our approach can be used to power a generative model over hierarchical data using hyperbolic latent variables.

Preprint, work under review

References in corpus (25)

• Adam: A Method for Stochastic Optimization
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Variational Inference with Normalizing Flows
• Normalizing Flows: An Introduction and Review of Current Methods
• Variational Graph Auto-Encoders
• Revisiting Semi-Supervised Learning with Graph Embeddings
• Wasserstein Auto-Encoders
• Efficient Graph Generation with Graph Recurrent Attention Networks
• FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
• Hyperspherical Variational Auto-Encoders
• Hyperbolic Graph Neural Networks
• Normalizing Flows on Tori and Spheres
• Neural Embeddings of Graphs in Hyperbolic Space
• Gradient Estimators for Implicit Models
• A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning
• Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders
• Hyperbolic Attention Networks
• Normalizing Flows on Riemannian Manifolds
• A Spectral Approach to Gradient Estimation for Implicit Distributions
• Graph Normalizing Flows
• Flows for simultaneous manifold learning and density estimation
• Adversarial Autoencoders with Constant-Curvature Latent Manifolds
• Block Neural Autoregressive Flow
• Poincaré Wasserstein Autoencoder
• Kernel Implicit Variational Inference

Cited by in corpus (8)

• Normalizing Flows: An Introduction and Review of Current Methods
• Riemannian Continuous Normalizing Flows
• Neural Manifold Ordinary Differential Equations
• Neural Ordinary Differential Equations on Manifolds
• Hyperbolic Deep Neural Networks: A Survey
• Moser Flow: Divergence-based Generative Modeling on Manifolds
• Principled Interpolation in Normalizing Flows
• Continuous normalizing flows on manifolds