Path Integral Sampler: a stochastic control approach for sampling
arXiv:2111.15141
Abstract
We present Path Integral Sampler~(PIS), a novel algorithm to draw samples from unnormalized probability density functions. The PIS is built on the Schrödinger bridge problem which aims to recover the most likely evolution of a diffusion process given its initial distribution and terminal distribution. The PIS draws samples from the initial distribution and then propagates the samples through the Schrödinger bridge to reach the terminal distribution. Applying the Girsanov theorem, with a simple prior diffusion, we formulate the PIS as a stochastic optimal control problem whose running cost is the control energy and terminal cost is chosen according to the target distribution. By modeling the control as a neural network, we establish a sampling algorithm that can be trained end-to-end. We provide theoretical justification of the sampling quality of PIS in terms of Wasserstein distance when sub-optimal control is used. Moreover, the path integrals theory is used to compute importance weights of the samples to compensate for the bias induced by the sub-optimality of the controller and time-discretization. We experimentally demonstrate the advantages of PIS compared with other start-of-the-art sampling methods on a variety of tasks.
iclr submission
References in corpus (14)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
- Learning in Implicit Generative Models
- Scalable Gradients for Stochastic Differential Equations
- i-flow: High-dimensional Integration and Sampling with Normalizing Flows
- NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport
- Learning Energy-Based Models by Diffusion Recovery Likelihood
- Gradient-based Adaptive Markov Chain Monte Carlo
- Deep Involutive Generative Models for Neural MCMC
- Computational Separations between Sampling and Optimization
- Schr{ö}dinger-F{ö}llmer Sampler: Sampling without Ergodicity
- A Neural Network MCMC sampler that maximizes Proposal Entropy
- Annealed Flow Transport Monte Carlo
- Efficient and Accurate Gradients for Neural SDEs