Deep Diffeomorphic Normalizing Flows

Abstract

The Normalizing Flow (NF) models a general probability density by estimating an invertible transformation applied on samples drawn from a known distribution. We introduce a new type of NF, called Deep Diffeomorphic Normalizing Flow (DDNF). A diffeomorphic flow is an invertible function where both the function and its inverse are smooth. We construct the flow using an ordinary differential equation (ODE) governed by a time-varying smooth vector field. We use a neural network to parametrize the smooth vector field and a recursive neural network (RNN) for approximating the solution of the ODE. Each cell in the RNN is a residual network implementing one Euler integration step. The architecture of our flow enables efficient likelihood evaluation, straightforward flow inversion, and results in highly flexible density estimation. An end-to-end trained DDNF achieves competitive results with state-of-the-art methods on a suite of density estimation and variational inference tasks. Finally, our method brings concepts from Riemannian geometry that, we believe, can open a new research direction for neural density estimation.

References in corpus (4)

• PDE-Net: Learning PDEs from Data
• Sylvester Normalizing Flows for Variational Inference
• Improving Variational Auto-Encoders using Householder Flow
• Pathwise Derivatives for Multivariate Distributions

Cited by in corpus (9)

• Normalizing Flows: An Introduction and Review of Current Methods
• CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations
• Riemannian Continuous Normalizing Flows
• Neural Spline Flows
• Confusion of Tagged Perturbations in Forward Automatic Differentiation of Higher-Order Functions
• Particle-based Energetic Variational Inference
• Neural Density Estimation and Likelihood-free Inference
• Gradient Boosted Normalizing Flows
• Accelerating Continuous Normalizing Flow with Trajectory Polynomial Regularization