Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks

Jim Zhao, Sidak Pal Singh, Aurelien Lucchi

The Gauss-Newton (GN) matrix plays an important role in machine learning, most evident in its use as a preconditioning matrix for a wide family of popular adaptive methods to speed…