Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators

Alexander Shekhovtsov

Discrete and especially binary random variables occur in many machine learning models, notably in variational autoencoders with binary latent states and in stochastic binary networ…