8 citations · 23 across the 12 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.LG2023★ 1 cited
Noise Balance and Stationary Distribution of Stochastic Gradient Descent
Liu Ziyin, Hongchao Li, Masahito Ueda
The stochastic gradient descent (SGD) algorithm is the algorithm we use to train neural networks. However, it remains poorly understood how the SGD navigates the highly nonlinear a…
physics.comp-ph2023
Pink-noise dynamics in an evolutionary game on a regular graph
Yuki Sakamoto, Masahito Ueda
We consider an iterated multiplayer prisoner's dilemma game on a square lattice and regular graphs based on the pairwise-Fermi update rule, and obtain heat-maps of the fraction of…
cs.LG2023★ 3 cited
Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent
Liu Ziyin, Botao Li, Tomer Galanti +1
Characterizing and understanding the dynamics of stochastic gradient descent (SGD) around saddle points remains an open problem. We first show that saddle points in neural networks…