Improved Gradient-Based Optimization Over Discrete Distributions
arXiv:1810.00116
Abstract
In many applications we seek to maximize an expectation with respect to a distribution over discrete variables. Estimating gradients of such objectives with respect to the distribution parameters is a challenging problem. We analyze existing solutions including finite-difference (FD) estimators and continuous relaxation (CR) estimators in terms of bias and variance. We show that the commonly used Gumbel-Softmax estimator is biased and propose a simple method to reduce it. We also derive a simpler piece-wise linear continuous relaxation that also possesses reduced bias. We demonstrate empirically that reduced bias leads to a better performance in variational inference and on binary optimization tasks.
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Categorical Reparameterization with Gumbel-Softmax
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Gradient Estimation Using Stochastic Computation Graphs
- REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
- Variational inference for Monte Carlo objectives
- Backpropagation through the Void: Optimizing control variates for black-box gradient estimation
- Discrete Variational Autoencoders
- Techniques for Learning Binary Stochastic Feedforward Neural Networks
- DVAE++: Discrete Variational Autoencoders with Overlapping Transformations
- ARM: Augment-REINFORCE-Merge Gradient for Stochastic Binary Networks
- MuProp: Unbiased Backpropagation for Stochastic Neural Networks
- DVAE#: Discrete Variational Autoencoders with Relaxed Boltzmann Priors
- Direct Optimization through for Discrete Variational Auto-Encoder