462 citations · 2.5k across the 127 of their papers we have counts for
11 papers · 1 filter
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
Difan Zou, Yuan Cao, Dongruo Zhou +1
We study the problem of training deep neural networks with Rectified Linear Unit (ReLU) activation function using gradient descent and stochastic gradient descent. In particular, w…
Sample Efficient Stochastic Variance-Reduced Cubic Regularization Method
Dongruo Zhou, Pan Xu, Quanquan Gu
We propose a sample efficient stochastic variance-reduced cubic regularization (Lite-SVRC) algorithm for finding the local minimum efficiently in nonconvex optimization. The propos…
A Frank-Wolfe Framework for Efficient and Effective Adversarial Attacks
Jinghui Chen, Dongruo Zhou, Jinfeng Yi +1
Depending on how much information an adversary can access to, adversarial attacks can be classified as white-box attack and black-box attack. For white-box attack, optimization-bas…
On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
Dongruo Zhou, Jinghui Chen, Yuan Cao +2
Adaptive gradient methods are workhorses in deep learning. However, the convergence guarantees of adaptive gradient methods for nonconvex optimization have not been thoroughly stud…
Finding Local Minima via Stochastic Nested Variance Reduction
Dongruo Zhou, Pan Xu, Quanquan Gu
We propose two algorithms that can find local minima faster than the state-of-the-art algorithms in both finite-sum and general stochastic nonconvex optimization. At the core of th…
Learning One-hidden-layer ReLU Networks via Gradient Descent
Xiao Zhang, Yaodong Yu, Lingxiao Wang +1
We study the problem of learning one-hidden-layer neural networks with Rectified Linear Unit (ReLU) activation function, where the inputs are sampled from standard Gaussian distrib…