Beyond ${\ell }_2$-norm and ${\ell }_{\infty }$-norm: A Curvature-Inspired ${\ell }_p$-Norm Scheme for Deep Neural Networks

Jianhao Xu, Zhuang Yang

The existing optimizers for deep neural networks (DNNs) typically rely on either the ${\ell }_2$ norm or the ${\ell }_{\infty }$ norm, resulting in optimizers that do not adapt well to subs…