Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks

Yikai Wu, Xingyu Zhu, Chenwei Wu +2

Hessian captures important properties of the deep neural network loss landscape. Previous works have observed low rank structure in the Hessians of neural networks. In this paper,…