4 citations · 11 across the 5 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2022★ 4 cited
A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks
Yuxin Sun, Dong Lao, Ganesh Sundaramoorthi +1
We discover restrained numerical instabilities in current training practices of deep networks with stochastic gradient descent (SGD), and its variants. We show numerical error (on…
cs.LG2020★ 1 cited
Channel-Directed Gradients for Optimization of Convolutional Neural Networks
Dong Lao, Peihao Zhu, Peter Wonka +1
We introduce optimization methods for convolutional neural networks that can be used to improve existing gradient-based optimization in terms of generalization error. The method re…