Rao-Blackwellized Stochastic Gradients for Discrete Distributions

Abstract

We wish to compute the gradient of an expectation over a finite or countably infinite sample space having $K\le \infty$ categories. When $K$ is indeed infinite, or finite but very large, the relevant summation is intractable. Accordingly, various stochastic gradient estimators have been proposed. In this paper, we describe a technique that can be applied to reduce the variance of any such estimator, without changing its bias---in particular, unbiasedness is retained. We show that our technique is an instance of Rao-Blackwellization, and we demonstrate the improvement it yields on a semi-supervised classification problem and a pixel attention task.

arXiv commentAccepted to ICML 2019

Cited by in corpus (9)

• Estimating Gradients for Discrete Random Variables by Sampling without Replacement
• Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
• Storchastic: A Framework for General Stochastic Automatic Differentiation
• Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions
• Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes
• Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator
• CARMS: Categorical-Antithetic-REINFORCE Multi-Sample Gradient Estimator
• Conditional Poisson Stochastic Beam Search
• Coordinate-wise Control Variates for Deep Policy Gradients