Path Integral Sampler: a stochastic control approach for sampling
arXiv:2111.15141
Abstract
We present Path Integral Sampler~(PIS), a novel algorithm to draw samples from unnormalized probability density functions. The PIS is built on the Schrödinger bridge problem which aims to recover the most likely evolution of a diffusion process given its initial distribution and terminal distribution. The PIS draws samples from the initial distribution and then propagates the samples through the Schrödinger bridge to reach the terminal distribution. Applying the Girsanov theorem, with a simple prior diffusion, we formulate the PIS as a stochastic optimal control problem whose running cost is the control energy and terminal cost is chosen according to the target distribution. By modeling the control as a neural network, we establish a sampling algorithm that can be trained end-to-end. We provide theoretical justification of the sampling quality of PIS in terms of Wasserstein distance when sub-optimal control is used. Moreover, the path integrals theory is used to compute importance weights of the samples to compensate for the bias induced by the sub-optimality of the controller and time-discretization. We experimentally demonstrate the advantages of PIS compared with other start-of-the-art sampling methods on a variety of tasks.
arXiv commenticlr submission
References in corpus (27)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Neural Tangent Kernel: Convergence and Generalization in Neural Networks
- Variational Inference with Normalizing Flows
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
- Density estimation using Real NVP
- Neural Ordinary Differential Equations
- Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
- Learning in Implicit Generative Models
- Residual Flows for Invertible Generative Modeling
- Scalable Gradients for Stochastic Differential Equations
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Asymptotically unbiased estimation of physical observables with neural samplers
- i-flow: High-dimensional Integration and Sampling with Normalizing Flows
- Measuring Sample Quality with Kernels
- A-NICE-MC: Adversarial Training for MCMC
- NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport
- Path Integral Control and State Dependent Feedback
- Understanding and Accelerating Particle-Based Variational Inference
- Learning Energy-Based Models by Diffusion Recovery Likelihood
- Gradient-based Adaptive Markov Chain Monte Carlo
- Deep Involutive Generative Models for Neural MCMC
- Computational Separations between Sampling and Optimization
- Schr{ö}dinger-F{ö}llmer Sampler: Sampling without Ergodicity
- Annealed Flow Transport Monte Carlo
- A Neural Network MCMC sampler that maximizes Proposal Entropy
- Efficient and Accurate Gradients for Neural SDEs