SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence
arXiv:2006.02509
Abstract
Stein Variational Gradient Descent (SVGD), a popular sampling algorithm, is often described as the kernelized gradient flow for the Kullback-Leibler divergence in the geometry of optimal transport. We introduce a new perspective on SVGD that instead views SVGD as the (kernelized) gradient flow of the chi-squared divergence which, we show, exhibits a strong form of uniform exponential ergodicity under conditions as weak as a Poincaré inequality. This perspective leads us to propose an alternative to SVGD, called Laplacian Adjusted Wasserstein Gradient Descent (LAWGD), that can be implemented from the spectral decomposition of the Laplacian operator associated with the target density. We show that LAWGD exhibits strong convergence guarantees and good practical performance.
arXiv comment20 pages, 5 figures
References in corpus (10)
- Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
- Underdamped Langevin MCMC: A non-asymptotic analysis
- Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm
- Convergence of Langevin MCMC in KL-divergence
- Understanding and Accelerating Particle-Based Variational Inference
- Stein Variational Gradient Descent With Matrix-Valued Kernels
- A Stein variational Newton method
- Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices
- Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem
- Exponential decay of Rényi divergence under Fokker-Planck equations