Normalizing Flows on Riemannian Manifolds

Abstract

We consider the problem of density estimation on Riemannian manifolds. Density estimation on manifolds has many applications in fluid-mechanics, optics and plasma physics and it appears often when dealing with angular variables (such as used in protein folding, robot limbs, gene-expression) and in general directional statistics. In spite of the multitude of algorithms available for density estimation in the Euclidean spaces ${\mathbf{R}}^n$ that scale to large n (e.g. normalizing flows, kernel methods and variational approximations), most of these methods are not immediately suitable for density estimation in more general Riemannian manifolds. We revisit techniques related to homeomorphisms from differential geometry for projecting densities to sub-manifolds and use it to generalize the idea of normalizing flows to more general Riemannian manifolds. The resulting algorithm is scalable, simple to implement and suitable for use with automatic differentiation. We demonstrate concrete examples of this method on the n-sphere ${\mathbf{S}}^n$.

3 pages, 2 figures, Submitted to Workshop on Bayesian Deep Learning at NIPS 2016

References in corpus (7)

• Auto-Encoding Variational Bayes
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Auxiliary Deep Generative Models
• Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
• Stochastic Backpropagation through Mixture Density Distributions
• Riemannian Manifold Hamiltonian Monte Carlo

Cited by in corpus (24)

• Normalizing Flows: An Introduction and Review of Current Methods
• Sampling using $SU(N)$ gauge equivariant flows
• Normalizing Flows on Tori and Spheres
• Efficient Modelling of Trivializing Maps for Lattice ${\varphi }^4$ Theory Using Normalizing Flows: A First Look at Scalability
• SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds
• Equivariant Hamiltonian Flows
• Riemannian Continuous Normalizing Flows
• CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations
• CaloMan: Fast generation of calorimeter showers with density estimation on learned manifolds
• Diffusion Variational Autoencoders
• Riemannian Stein Variational Gradient Descent for Bayesian Inference
• Introduction to Normalizing Flows for Lattice Field Theory
• Neural Ordinary Differential Equations on Manifolds
• Hyperbolic Deep Neural Networks: A Survey
• Latent Variable Modelling with Hyperbolic Normalizing Flows
• Normalizing Flows Across Dimensions
• Principled Interpolation in Normalizing Flows
• Refining Deep Generative Models via Discriminator Gradient Flow
• Rectangular Flows for Manifold Learning
• Interpreting diffusion score matching using normalizing flow
• Ordering Dimensions with Nested Dropout Normalizing Flows
• Spherical Sliced-Wasserstein
• Reparameterized Sampling for Generative Adversarial Networks
• Continuous normalizing flows on manifolds