1.3k citations · 1.7k across the 25 of their papers we have counts for
4 papers · 1 filter
Feature Learning in -regularized DNNs: Attraction/Repulsion and Sparsity
Arthur Jacot, Eugene Golikov, Clément Hongler +1
We study the loss surface of DNNs with regularization. We show that the loss in terms of the parameters can be reformulated into a loss in terms of the layerwise activation…
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity
Arthur Jacot, François Ged, Berfin Şimşek +2
The dynamics of Deep Linear Networks (DLNs) is dramatically affected by the variance of the parameters at initialization . For DLNs of width , we show a phase transit…
Kernel Alignment Risk Estimator: Risk Prediction from Training Data
Arthur Jacot, Berfin Şimşek, Francesco Spadaro +2
We study the risk (i.e. generalization error) of Kernel Ridge Regression (KRR) for a kernel with ridge and i.i.d. observations. For this, we introduce two objects: the Si…
Implicit Regularization of Random Feature Models
Arthur Jacot, Berfin Şimşek, Francesco Spadaro +2
Random Feature (RF) models are used as efficient parametric approximations of kernel methods. We investigate, by means of random matrix theory, the connection between Gaussian RF m…