Stein Variational Gradient Descent Without Gradient

Abstract

Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for complex distributions. However, the standard SVGD requires calculating the gradient of the target density and cannot be applied when the gradient is unavailable. In this work, we develop a gradient-free variant of SVGD (GF-SVGD), which replaces the true gradient with a surrogate gradient, and corrects the induced bias by re-weighting the gradients in a proper form. We show that our GF-SVGD can be viewed as the standard SVGD with a special choice of kernel, and hence directly inherits the theoretical properties of SVGD. We shed insights on the empirical choice of the surrogate gradient and propose an annealed GF-SVGD that leverages the idea of simulated annealing to improve the performance on high dimensional complex distributions. Empirical studies show that our method consistently outperforms a number of recent advanced gradient-free MCMC methods.

arXiv commentICML 2018

References in corpus (7)

• Stein Variational Gradient Descent as Gradient Flow
• Measuring Sample Quality with Stein's Method
• Measuring Sample Quality with Kernels
• Stein Variational Policy Gradient
• Kernel Adaptive Metropolis-Hastings
• Advances in Variational Inference
• Stein Variational Adaptive Importance Sampling

Cited by in corpus (11)

• Stein Variational Gradient Descent With Matrix-Valued Kernels
• Stein variational gradient descent with local approximations
• Oops I Took A Gradient: Scalable Sampling for Discrete Distributions
• Stein Self-Repulsive Dynamics: Benefits From Past Samples
• Stein Variational Inference for Discrete Distributions
• Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimization
• Doubly Robust Stein-Kernelized Monte Carlo Estimator: Simultaneous Bias-Variance Reduction and Supercanonical Convergence
• On Unifying Randomized Methods For Inverse Problems
• Two Stochastic Control Problems In Capital Structure and Portfolio Choice
• Scalable Approximate Inference and Some Applications
• Structure-Aware Human-Action Generation