Particle-based Energetic Variational Inference
arXiv:2004.06443 · doi:10.1007/s11222-021-10009-7
Abstract
We introduce a new variational inference (VI) framework, called energetic variational inference (EVI). It minimizes the VI objective function based on a prescribed energy-dissipation law. Using the EVI framework, we can derive many existing Particle-based Variational Inference (ParVI) methods, including the popular Stein Variational Gradient Descent (SVGD) approach. More importantly, many new ParVI schemes can be created under this framework. For illustration, we propose a new particle-based EVI scheme, which performs the particle-based approximation of the density first and then uses the approximated density in the variational procedure, or "Approximation-then-Variation" for short. Thanks to this order of approximation and variation, the new scheme can maintain the variational structure at the particle level, and can significantly decrease the KL-divergence in each iteration. Numerical experiments show the proposed method outperforms some existing ParVI methods in terms of fidelity to the target distribution.
17 pages, 7 figures
References in corpus (13)
- Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
- Normalizing Flows for Probabilistic Modeling and Inference
- Maximum Mean Discrepancy Gradient Flow
- Nonparametric variational inference
- Understanding and Accelerating Particle-Based Variational Inference
- Stein Variational Gradient Descent With Matrix-Valued Kernels
- A Stein variational Newton method
- On Lagrangian schemes for porous medium type generalized diffusion equations: a discrete energetic variational approach
- A stochastic version of Stein Variational Gradient Descent for efficient sampling
- Laplacian Smoothing Gradient Descent
- Approximate inference with Wasserstein gradient flows
- Deep Diffeomorphic Normalizing Flows
- A Variational Lagrangian Scheme for a Phase Field Model: A Discrete Energetic Variational Approach