1 paper · 1 filter
Benjamin Gess, Daniel Heydecker
Large loss spikes in stochastic gradient descent are studied through a rigorous large-deviations analysis for a shallow, fully connected network in the NTK scaling. In contrast to…