212 citations · 799 across the 27 of their papers we have counts for
4 papers · 1 filter
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
Difan Zou, Yuan Cao, Dongruo Zhou +1
We study the problem of training deep neural networks with Rectified Linear Unit (ReLU) activation function using gradient descent and stochastic gradient descent. In particular, w…
High-Temperature Structure Detection in Ferromagnets
Yuan Cao, Matey Neykov, Han Liu
This paper studies structure detection problems in high temperature ferromagnetic (positive interaction only) Ising models. The goal is to distinguish whether the underlying graph…
On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
Dongruo Zhou, Jinghui Chen, Yuan Cao +2
Adaptive gradient methods are workhorses in deep learning. However, the convergence guarantees of adaptive gradient methods for nonconvex optimization have not been thoroughly stud…
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
Jinghui Chen, Dongruo Zhou, Yiqi Tang +3
Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed…