CARMS: Categorical-Antithetic-REINFORCE Multi-Sample Gradient Estimator

Abstract

Accurately backpropagating the gradient through categorical variables is a challenging task that arises in various domains, such as training discrete latent variable models. To this end, we propose CARMS, an unbiased estimator for categorical random variables based on multiple mutually negatively correlated (jointly antithetic) samples. CARMS combines REINFORCE with copula based sampling to avoid duplicate samples and reduce its variance, while keeping the estimator unbiased using importance sampling. It generalizes both the ARMS antithetic estimator for binary variables, which is CARMS for two categories, as well as LOORF/VarGrad, the leave-one-out REINFORCE estimator, which is CARMS with independent samples. We evaluate CARMS on several benchmark datasets on a generative modeling task, as well as a structured output prediction task, and find it to outperform competing methods including a strong self-control baseline. The code is publicly available.

References in corpus (17)

• Auto-Encoding Variational Bayes
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Categorical Reparameterization with Gumbel-Softmax
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Zero-Shot Text-to-Image Generation
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Neural Variational Inference and Learning in Belief Networks
• REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
• Quantifying the Carbon Emissions of Machine Learning
• Variational Bayesian Inference with Stochastic Search
• Stochastic Optimization of Sorting Networks via Continuous Relaxations
• On Using Control Variates with Stochastic Approximation for Variational Bayes and its Connection to Stochastic Linear Regression
• Rao-Blackwellized Stochastic Gradients for Discrete Distributions
• Estimating Gradients for Discrete Random Variables by Sampling without Replacement
• ARSM: Augment-REINFORCE-Swap-Merge Estimator for Gradient Backpropagation Through Categorical Variables
• ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variables
• VarGrad: A Low-Variance Gradient Estimator for Variational Inference