Monge-Ampère Flow for Generative Modeling

Abstract

We present a deep generative model, named Monge-Ampère flow, which builds on continuous-time gradient flow arising from the Monge-Ampère equation in optimal transport theory. The generative map from the latent space to the data space follows a dynamical system, where a learnable potential function guides a compressible fluid to flow towards the target density distribution. Training of the model amounts to solving an optimal control problem. The Monge-Ampère flow has tractable likelihoods and supports efficient sampling and inference. One can easily impose symmetry constraints in the generative model by designing suitable scalar potential functions. We apply the approach to unsupervised density estimation of the MNIST dataset and variational calculation of the two-dimensional Ising model at the critical point. This approach brings insights and techniques from Monge-Ampère equation, optimal transport, and fluid dynamics into reversible flow-based generative models.

References in corpus (6)

• NICE: Non-linear Independent Components Estimation
• MADE: Masked Autoencoder for Distribution Estimation
• Deep Learning Approximation for Stochastic Control Problems
• Reversible Architectures for Arbitrarily Deep Residual Neural Networks
• GAN and VAE from an Optimal Transport Point of View
• A Flow Model of Neural Networks

Cited by in corpus (34)

• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• Flow-based generative models for Markov chain Monte Carlo in lattice field theory
• Equivariant flow-based sampling for lattice gauge theory
• Sampling using $SU(N)$ gauge equivariant flows
• How to train your neural ODE: the world of Jacobian and kinetic regularization
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities
• Adaptive deep density approximation for Fokker-Planck equations
• CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations
• Differentiable Molecular Simulations for Control and Learning
• Ab-initio study of interacting fermions at finite temperature with neural canonical transformation
• Neural Canonical Transformation with Symplectic Flows
• Learning Neural Event Functions for Ordinary Differential Equations
• Stochastic Normalizing Flows
• Exponential Family Estimation via Adversarial Dynamics Embedding
• Coupling the reduced-order model and the generative model for an importance sampling estimator
• Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
• Neural Density Estimation and Likelihood-free Inference
• Generalization and Memorization: The Bias Potential Model
• On the Need for Topology-Aware Generative Models for Manifold-Based Defenses
• Temperature-steerable flows
• Learning normalizing flows from Entropy-Kantorovich potentials
• Learning Implicit Generative Models with Theoretical Guarantees
• VAE-KRnet and its applications to variational Bayes
• A Mathematical Framework for Learning Probability Distributions
• Characteristic Neural Ordinary Differential Equations
• Generative Learning With Euler Particle Transport
• Jacobian Determinant of Normalizing Flows
• Training Invertible Linear Layers through Rank-One Perturbations
• Augmented KRnet for density estimation and approximation
• A Triangular Network For Density Estimation
• EBMs Trained with Maximum Likelihood are Generator Models Trained with a Self-adverserial Loss
• Review: Ordinary Differential Equations For Deep Learning
• Particle Flow Bayes' Rule