Continuous normalizing flows on manifolds

Abstract

Normalizing flows are a powerful technique for obtaining reparameterizable samples from complex multimodal distributions. Unfortunately, current approaches are only available for the most basic geometries and fall short when the underlying space has a nontrivial topology, limiting their applicability for most real-world data. Using fundamental ideas from differential geometry and geometric control theory, we describe how the recently introduced Neural ODEs and continuous normalizing flows can be extended to arbitrary smooth manifolds. We propose a general methodology for parameterizing vector fields on these spaces and demonstrate how gradient-based learning can be performed. Additionally, we provide a scalable unbiased estimator for the divergence in this generalized setting. Experiments on a diverse selection of spaces empirically showcase the defined framework's ability to obtain reparameterizable samples from complex distributions.

arXiv commentMaster Thesis

References in corpus (20)

• Auto-Encoding Variational Bayes
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Normalizing Flows: An Introduction and Review of Current Methods
• Normalizing Flows for Probabilistic Modeling and Inference
• Equivariant flow-based sampling for lattice gauge theory
• Implicit Reparameterization Gradients
• Sampling using $SU(N)$ gauge equivariant flows
• Normalizing Flows on Tori and Spheres
• Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders
• A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning
• Normalizing Flows on Riemannian Manifolds
• Riemannian Continuous Normalizing Flows
• Poisson structures on the cotangent bundle of a Lie group or a principal bundle and their reductions
• Diffusion Variational Autoencoders
• Neural Manifold Ordinary Differential Equations
• The Power Spherical distribution
• Latent Variable Modelling with Hyperbolic Normalizing Flows
• Laplace Transformations of Submanifolds
• Increasing Expressivity of a Hyperspherical VAE