A note on regularised NTK dynamics with an application to PAC-Bayesian training

Eugenio Clerico, Benjamin Guedj

We establish explicit dynamics for neural networks whose training objective has a regularising term that constrains the parameters to remain close to their initial value. This keep…