1.3k citations · 1.6k across the 25 of their papers we have counts for
4 papers · 1 filter
Understanding Layer-wise Contributions in Deep Neural Networks through Spectral Analysis
Yatin Dandi, Arthur Jacot
Spectral analysis is a powerful tool, decomposing any function into simpler parts. In machine learning, Mercer's theorem generalizes this idea, providing for any kernel and input d…
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity
Arthur Jacot, François Ged, Berfin Şimşek +2
The dynamics of Deep Linear Networks (DLNs) is dramatically affected by the variance of the parameters at initialization . For DLNs of width , we show a phase transit…
DNN-Based Topology Optimisation: Spatial Invariance and Neural Tangent Kernel
Benjamin Dupuis, Arthur Jacot
We study the Solid Isotropic Material Penalisation (SIMP) method with a density field generated by a fully-connected neural network, taking the coordinates as inputs. In the large…
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances
Berfin Şimşek, François Ged, Arthur Jacot +4
We study how permutation symmetries in overparameterized multi-layer neural networks generate `symmetry-induced' critical points. Assuming a network with layers of minimal wi…