Training Neural Networks with Stochastic Hessian-Free Optimization

Ryan Kiros

Hessian-free (HF) optimization has been successfully used for training deep autoencoders and recurrent networks. HF uses the conjugate gradient algorithm to construct update direct…