204 citations · 848 across the 22 of their papers we have counts for
4 papers · 1 filter
An Exponential Learning Rate Schedule for Deep Learning
Zhiyuan Li, Sanjeev Arora
Intriguing empirical evidence exists that deep learning can work well with exoticschedules for varying the learning rate. This paper suggests that the phenomenon may be due to Batc…
Enhanced Convolutional Neural Tangent Kernels
Zhiyuan Li, Ruosong Wang, Dingli Yu +4
Recent research shows that for training with loss, convolutional neural networks (CNNs) whose width (number of channels in convolutional layers) goes to infinity correspon…
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks
Sanjeev Arora, Simon S. Du, Zhiyuan Li +3
Recent research shows that the following two models are equivalent: (a) infinitely wide neural networks (NNs) trained under l2 loss by gradient descent with infinitesimally small l…
A Theoretical Analysis of Contrastive Unsupervised Representation Learning
Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak +2
Recent empirical works have successfully used unlabeled data to learn feature representations that are broadly useful in downstream classification tasks. Several of these methods a…