3 papers
cs.LG2025
Noise Balance and Stationary Distribution of Stochastic Gradient Descent
Liu Ziyin, Hongchao Li, Masahito Ueda
The stochastic gradient descent (SGD) algorithm is the algorithm we use to train neural networks. However, it remains poorly understood how the SGD navigates the highly nonlinear a…
cs.LG2024
Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent
Liu Ziyin, Botao Li, Tomer Galanti +1
Characterizing and understanding the dynamics of stochastic gradient descent (SGD) around saddle points remains an open problem. We first show that saddle points in neural networks…
physics.comp-ph2024
Pink-noise dynamics in an evolutionary game on a regular graph
Yuki Sakamoto, Masahito Ueda
We consider an iterated multiplayer prisoner's dilemma game on a square lattice and regular graphs based on the pairwise-Fermi update rule, and obtain heat-maps of the fraction of…