Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks

Alexander Shekhovtsov, Viktor Yanush, Boris Flach

In neural networks with binary activations and or binary weights the training by gradient descent is complicated as the model has piecewise constant response. We consider stochasti…