Training Integrable Parameterizations of Deep Neural Networks in the Infinite-Width Limit

Karl Hajjar, Lénaïc Chizat, Christophe Giraud

To theoretically understand the behavior of trained deep neural networks, it is necessary to study the dynamics induced by gradient methods from a random initialization. However, t…