Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient

Vu C. Dinh, Lam Si Tung Ho, Cuong V. Nguyen

We analyze the error rates of the Hamiltonian Monte Carlo algorithm with leapfrog integrator for Bayesian neural network inference. We show that due to the non-differentiability of…