Analytic Insights into Structure and Rank of Neural Network Hessian Maps

Sidak Pal Singh, Gregor Bachmann, Thomas Hofmann

The Hessian of a neural network captures parameter interactions through second-order derivatives of the loss. It is a fundamental object of study, closely tied to various problems…