Singular Learning and Occam's Razor in Deep Monomial Networks

Kathlén Kohn, Giovanni Luca Marchetti, Farhan Shabir +2

In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture. These critical points occur where the Jacobian…