Stochastic Normalizing Flows
arXiv:2002.09547
Abstract
We introduce stochastic normalizing flows, an extension of continuous normalizing flows for maximum likelihood estimation and variational inference (VI) using stochastic differential equations (SDEs). Using the theory of rough paths, the underlying Brownian motion is treated as a latent variable and approximated, enabling efficient training of neural SDEs as random neural ordinary differential equations. These SDEs can be used for constructing efficient Markov chains to sample from the underlying distribution of a given dataset. Furthermore, by considering families of targeted SDEs with prescribed stationary distribution, we can apply VI to the optimization of hyperparameters in stochastic MCMC.
17 pages, 4 figures
References in corpus (13)
- Variational Inference with Normalizing Flows
- Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
- Scalable Gradients for Stochastic Differential Equations
- Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise
- Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit
- ANODEV2: A Coupled Neural ODE Evolution Framework
- Black-box Variational Inference for Stochastic Differential Equations
- Unbiased Implicit Variational Inference
- Variational Inference with Hamiltonian Monte Carlo
- Unbiased estimation of log normalizing constants with applications to Bayesian cross-validation
- Two Methods For Wild Variational Inference
- Implicit Langevin Algorithms for Sampling From Log-concave Densities
- Infinitely deep neural networks as diffusion processes
Cited by in corpus (11)
- Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU
- normflows: A PyTorch Package for Normalizing Flows
- Learning the temporal evolution of multivariate densities via normalizing flows
- STEER: Simple Temporal Regularization For Neural ODEs
- Stochastic Normalizing Flows
- Text to Image Synthesis using Stacked Conditional Variational Autoencoders and Conditional Generative Adversarial Networks
- Normalizing flows as an enhanced sampling method for atomistic supercooled liquids
- Neural SDEs as Infinite-Dimensional GANs
- Computing Absolute Free Energy with Deep Generative Models
- Noisy Recurrent Neural Networks
- E2V-SDE: From Asynchronous Events to Fast and Continuous Video Reconstruction via Neural Stochastic Differential Equations