7 papers
Spectral phase transitions and trainability in neural network learning dynamics
Chanju Park, Dario Bocchi, Francesco D'Amico +2
The emergence of low-dimensional structures in the spectra of neural network weight matrices is a common empirical feature of trained models, but the dynamical origin of this pheno…
Stochastic Path Sampler For Lattice Field Theory
Shiyang Chen, Moxian Qian, Gert Aarts +2
In lattice field theory, target distributions are known only up to normalization, (\tildeÏ(Ï)\propto e^{-S(Ï)}), while the partition function is intractable. Markov chain Monte…
Phase diagram and eigenvalue dynamics of stochastic gradient descent in multilayer neural networks
Chanju Park, Biagio Lucini, Gert Aarts
Hyperparameter tuning is one of the essential steps to guarantee the convergence of machine learning models. We argue that intuition about the optimal choice of hyperparameters for…
Exploring Generative Networks for Manifolds with Non-Trivial Topology
Shiyang Chen, Gert Aarts, Biagio Lucini
The expressive power of neural networks in modelling non-trivial distributions can in principle be exploited to bypass topological freezing and critical slowing down in simulations…
Random Matrix Theory for Stochastic Gradient Descent
Chanju Park, Matteo Favoni, Biagio Lucini +1
Investigating the dynamics of learning in machine learning algorithms is of paramount importance for understanding how and why an approach may be successful. The tools of physics a…
Stochastic weight matrix dynamics during learning and Dyson Brownian motion
Gert Aarts, Biagio Lucini, Chanju Park
We demonstrate that the update of weight matrices in learning algorithms can be described in the framework of Dyson Brownian motion, thereby inheriting many features of random matr…