Continuous-Time Flows for Efficient Inference and Density Estimation

Abstract

Two fundamental problems in unsupervised learning are efficient inference for latent-variable models and robust density estimation based on large amounts of unlabeled data. Algorithms for the two tasks, such as normalizing flows and generative adversarial networks (GANs), are often developed independently. In this paper, we propose the concept of {\em continuous-time flows} (CTFs), a family of diffusion-based methods that are able to asymptotically approach a target distribution. Distinct from normalizing flows and GANs, CTFs can be adopted to achieve the above two goals in one framework, with theoretical guarantees. Our framework includes distilling knowledge from a CTF for efficient inference, and learning an explicit energy-based distribution with CTFs for density estimation. Both tasks rely on a new technique for distribution matching within amortized learning. Experiments on various tasks demonstrate promising performance of the proposed CTF framework, compared to related techniques.

arXiv commentICML 2018 (fixed a reference)

Cited by in corpus (16)

• Normalizing Flows: An Introduction and Review of Current Methods
• ODE${}^2$VAE: Deep generative second order ODEs with Bayesian neural networks
• Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities
• Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit
• CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations
• Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics
• Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint
• Stochastic Normalizing Flows
• E(n) Equivariant Normalizing Flows
• Implicit Deep Latent Variable Models for Text Generation
• Implicit Normalizing Flows
• Learning Implicit Generative Models with Theoretical Guarantees
• Generative Learning With Euler Particle Transport
• Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs
• Gradient Boosted Normalizing Flows
• MCMC-Interactive Variational Inference