Reducing Reparameterization Gradient Variance

Abstract

Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the "reparameterization trick," represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimators are too noisy, the optimization procedure can be slow or fail to converge. One way to reduce noise is to use more samples for the gradient estimate, but this can be computationally expensive. Instead, we view the noisy gradient as a random variable, and form an inexpensive approximation of the generating procedure for the gradient sample. This approximation has high correlation with the noisy gradient by construction, making it a useful control variate for variance reduction. We demonstrate our approach on non-conjugate multi-level hierarchical models and a Bayesian neural net where we observed gradient variance reductions of multiple orders of magnitude (20-2,000x).

Cited by in corpus (24)

• Monte Carlo Gradient Estimation in Machine Learning
• Pathwise Derivatives Beyond the Reparameterization Trick
• The continuous Bernoulli: fixing a pervasive error in variational autoencoders
• A Tutorial on Deep Latent Variable Models of Natural Language
• Amortized Variational Inference: A Systematic Review
• Advances in Variational Inference
• Sampling-Free Variational Inference of Bayesian Neural Networks by Variance Backpropagation
• Differentiable Unsupervised Feature Selection based on a Gated Laplacian
• Faster Meta Update Strategy for Noise-Robust Deep Learning
• Hierarchical Gaussian Process Priors for Bayesian Neural Network Weights
• Variational Rejection Sampling
• Invertible Gaussian Reparameterization: Revisiting the Gumbel-Softmax
• New Tricks for Estimating Gradients of Expectations
• Variational Inference over Non-differentiable Cardiac Simulators using Bayesian Optimization
• Amortized variance reduction for doubly stochastic objectives
• Stochastic Gradient Variance Reduction by Solving a Filtering Problem
• Generalized Doubly Reparameterized Gradient Estimators
• Pathwise Derivatives for Multivariate Distributions
• A Rule for Gradient Estimator Selection, with an Application to Variational Inference
• Quasi-Newton Quasi-Monte Carlo for variational Bayes
• Approximation Based Variance Reduction for Reparameterization Gradients
• Multilevel Monte Carlo Variational Inference
• An adaptive mixture-population Monte Carlo method for likelihood-free inference
• Quantized Variational Inference