Coupled Gradient Estimators for Discrete Latent Variables
arXiv:2106.08056
Abstract
Training models with discrete latent variables is challenging due to the high variance of unbiased gradient estimators. While low-variance reparameterization gradients of a continuous relaxation can provide an effective solution, a continuous relaxation is not always available or tractable. Dong et al. (2020) and Yin et al. (2020) introduced a performant estimator that does not rely on continuous relaxations; however, it is limited to binary random variables. We introduce a novel derivation of their estimator based on importance sampling and statistical couplings, which we extend to the categorical setting. Motivated by the construction of a stick-breaking coupling, we introduce gradient estimators based on reparameterizing categorical variables as sequences of binary variables and Rao-Blackwellization. In systematic experiments, we show that our proposed categorical gradient estimators provide state-of-the-art performance, whereas even with additional Rao-Blackwellization, previous estimators (Yin et al., 2019) underperform a simpler REINFORCE with a leave-one-out-baseline estimator (Kool et al., 2019).
arXiv commentPublished in NeurIPS 2021
References in corpus (11)
- Auto-Encoding Variational Bayes
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Categorical Reparameterization with Gumbel-Softmax
- Empirical Evaluation of Rectified Activations in Convolutional Network
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
- Carbon Emissions and Large Neural Network Training
- ARM: Augment-REINFORCE-Merge Gradient for Stochastic Binary Networks
- Overdispersed Black-Box Variational Inference
- ARSM: Augment-REINFORCE-Swap-Merge Estimator for Gradient Backpropagation Through Categorical Variables
- DisARM: An Antithetic Gradient Estimator for Binary Latent Variables