61 citations · 750 across the 66 of their papers we have counts for
66 papers
Why Adam Can Beat SGD: Second-Moment Normalization Yields Sharper Tails
Ruinan Jin, Yingbin Liang, Shaofeng Zou
Despite Adam demonstrating faster empirical convergence than SGD in many applications, much of the existing theory yields guarantees essentially comparable to those of SGD, leaving…
Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization
Ziyi Chen, Yi Zhou, Yingbin Liang +1
Various optimal gradient-based algorithms have been developed for smooth nonconvex optimization. However, many nonconvex machine learning problems do not belong to the class of smo…
Pruning Before Training May Improve Generalization, Provably
Hongru Yang, Yingbin Liang, Xiaojie Guo +2
It has been observed in practice that applying pruning-at-initialization methods to neural networks and training the sparsified networks can not only retain the testing performance…
Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK
Hongru Yang, Ziyu Jiang, Ruizhe Zhang +2
We study training one-hidden-layer ReLU networks in the neural tangent kernel (NTK) regime, where the networks' biases are initialized to some constant rather than zero. We prove t…
Global Convergence of Two-timescale Actor-Critic for Solving Linear Quadratic Regulator
Xuyang Chen, Jingliang Duan, Yingbin Liang +1
The actor-critic (AC) reinforcement learning algorithms have been the powerhouse behind many challenging applications. Nevertheless, its convergence is fragile in general. To study…
Provable Generalization of Overparameterized Meta-learning Trained with SGD
Yu Huang, Yingbin Liang, Longbo Huang
Despite the superior empirical success of deep meta-learning, theoretical understanding of overparameterized meta-learning is still limited. This paper studies the generalization o…