5.2k citations · 7.8k across the 38 of their papers we have counts for
Showing 2017 · cs.LGShow all
2 papers · 2 filters
cs.LG2017★ 145 cited
Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
Levent Sagun, Utku Evci, V. Ugur Guney +2
We study the properties of common loss surfaces through their Hessian matrix. In particular, in the context of deep learning, we empirically show that the spectrum of the Hessian i…
cs.LG2017★ 6 cited
Diagonal Rescaling For Neural Networks
Jean Lafond, Nicolas Vasilache, Léon Bottou
We define a second-order neural network stochastic gradient training algorithm whose block-diagonal structure effectively amounts to normalizing the unit activations. Investigating…