Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theory

Abstract

Particle-optimization-based sampling (POS) is a recently developed effective sampling technique that interactively updates a set of particles. A representative algorithm is the Stein variational gradient descent (SVGD). We prove, under certain conditions, SVGD experiences a theoretical pitfall, {\it i.e.}, particles tend to collapse. As a remedy, we generalize POS to a stochastic setting by injecting random noise into particle updates, thus yielding particle-optimization sampling (SPOS). Notably, for the first time, we develop {\em non-asymptotic convergence theory} for the SPOS framework (related to SVGD), characterizing algorithm convergence in terms of the 1-Wasserstein distance w.r.t.\! the numbers of particles and iterations. Somewhat surprisingly, with the same number of updates (not too large) for each particle, our theory suggests adopting more particles does not necessarily lead to a better approximation of a target distribution, due to limited computational budget and numerical errors. This phenomenon is also observed in SVGD and verified via an experiment on synthetic data. Extensive experimental results verify our theory and demonstrate the effectiveness of our proposed framework.

arXiv commentAISTATS 2020

References in corpus (18)

• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Weight Uncertainty in Neural Networks
• VIME: Variational Information Maximizing Exploration
• Stochastic Gradient Hamiltonian Monte Carlo
• User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient
• A Complete Recipe for Stochastic Gradient MCMC
• Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
• On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators
• Stein Variational Policy Gradient
• Message Passing Stein Variational Gradient Descent
• A Unified Particle-Optimization Framework for Scalable Bayesian Sampling
• Stein Variational Gradient Descent as Moment Matching
• Stochastic Expectation Propagation
• On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo
• Riemannian Stein Variational Gradient Descent for Bayesian Inference
• Learning Structural Weight Uncertainty for Sequential Decision-Making
• Scalable Thompson Sampling via Optimal Transport
• Scaling limit of the Stein variational gradient descent: the mean field regime