On the Power-Law Hessian Spectrums in Deep Learning

Zeke Xie, Qian-Yuan Tang, Yunfeng Cai +2

It is well-known that the Hessian of deep loss landscape matters to optimization, generalization, and even robustness of deep learning. Recent works empirically discovered that the…