Markovian Score Climbing: Variational Inference with KL(p||q)

Abstract

Modern variational inference (VI) uses stochastic gradients to avoid intractable expectations, enabling large-scale probabilistic inference in complex models. VI posits a family of approximating distributions q and then finds the member of that family that is closest to the exact posterior p. Traditionally, VI algorithms minimize the "exclusive Kullback-Leibler (KL)" KL(q || p), often for computational convenience. Recent research, however, has also focused on the "inclusive KL" KL(p || q), which has good statistical properties that makes it more appropriate for certain inference problems. This paper develops a simple algorithm for reliably minimizing the inclusive KL using stochastic gradients with vanishing bias. This method, which we call Markovian score climbing (MSC), converges to a local optimum of the inclusive KL. It does not suffer from the systematic errors inherent in existing methods, such as Reweighted Wake-Sleep and Neural Adaptive Sequential Monte Carlo, which lead to bias in their final estimates. We illustrate convergence on a toy model and demonstrate the utility of MSC on Bayesian probit regression for classification as well as a stochastic volatility model for financial data.

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Expectation Propagation for approximate Bayesian inference
• The Measure and Mismeasure of Fairness
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Variational Bayesian Inference with Stochastic Search
• Auto-Encoding Sequential Monte Carlo
• NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport
• Rényi Divergence Variational Inference
• Approximate Inference with Amortised MCMC
• A Contrastive Divergence for Combining Variational Inference and MCMC
• Meta-Learning MCMC Proposals
• Reweighted Expectation Maximization
• Graphical model inference: Sequential Monte Carlo meets deterministic approximations
• Particle Smoothing Variational Objectives
• Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximation
• On importance-weighted autoencoders
• Learning dynamical systems with particle stochastic approximation EM

Cited by in corpus (5)

• Adaptive Monte Carlo augmented with normalizing flows
• Challenges and Opportunities in High-dimensional Variational Inference
• Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Models
• Energy-Based Models with Applications to Speech and Language Processing
• On the Difficulty of Unbiased Alpha Divergence Minimization