16 citations · 27 across the 17 of their papers we have counts for
4 papers · 1 filter
Spectral phase transitions and trainability in neural network learning dynamics
Chanju Park, Dario Bocchi, Francesco D'Amico +2
The emergence of low-dimensional structures in the spectra of neural network weight matrices is a common empirical feature of trained models, but the dynamical origin of this pheno…
Phase diagram and eigenvalue dynamics of stochastic gradient descent in multilayer neural networks
Chanju Park, Biagio Lucini, Gert Aarts
Hyperparameter tuning is one of the essential steps to guarantee the convergence of machine learning models. We argue that intuition about the optimal choice of hyperparameters for…
Dyson Brownian motion and random matrix dynamics of weight matrices during learning
Gert Aarts, Ouraman Hajizadeh, Biagio Lucini +1
During training, weight matrices in machine learning architectures are updated using stochastic gradient descent or variations thereof. In this contribution we employ concepts of r…
Stochastic weight matrix dynamics during learning and Dyson Brownian motion
Gert Aarts, Biagio Lucini, Chanju Park
We demonstrate that the update of weight matrices in learning algorithms can be described in the framework of Dyson Brownian motion, thereby inheriting many features of random matr…