75 citations · 145 across the 11 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.LG2021★ 4 cited
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion
Daniel Kunin, Javier Sagastuy-Brena, Lauren Gillespie +4
In this work we explore the limiting dynamics of deep neural networks trained with stochastic gradient descent (SGD). As observed previously, long after performance has converged,…
cs.LG2021★ 1 cited
Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks
Hidenori Tanaka, Daniel Kunin
In nature, symmetry governs regularities, while symmetry breaking brings texture. In artificial neural networks, symmetry has been a central design principle to efficiently capture…