Regularizing Deep Neural Networks with Stochastic Estimators of Hessian Trace

Yucong Liu, Shixing Yu, Tong Lin

In this paper, we develop a novel regularization method for deep neural networks by penalizing the trace of Hessian. This regularizer is motivated by a recent guarantee bound of th…