127 citations · 272 across the 4 of their papers we have counts for
4 papers
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK
Yuanzhi Li, Tengyu Ma, Hongyang R. Zhang
We consider the dynamic of gradient descent for learning a two-layer neural network. We assume the input is drawn from a Gaussian distribution and the label of $…
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks
Yuanzhi Li, Colin Wei, Tengyu Ma
Stochastic gradient descent with a large initial learning rate is widely used for training modern neural net architectures. Although a small initial learning rate allows for faster…
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees
Yuping Luo, Huazhe Xu, Yuanzhi Li +3
Model-based reinforcement learning (RL) is considered to be a promising approach to reduce the sample complexity that hinders model-free RL. However, the theoretical understanding…
Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations
Yuanzhi Li, Tengyu Ma, Hongyang Zhang
We show that the gradient descent algorithm provides an implicit regularization effect in the learning of over-parameterized matrix factorization models and one-hidden-layer neural…