From the 1 of 34 linked papers with an AI index.
256 citations · 486 across the 26 of their papers we have counts for
6 papers · 1 filter
An Exponential Learning Rate Schedule for Deep Learning
Zhiyuan Li, Sanjeev Arora
Intriguing empirical evidence exists that deep learning can work well with exoticschedules for varying the learning rate. This paper suggests that the phenomenon may be due to Batc…
Enhanced Convolutional Neural Tangent Kernels
Zhiyuan Li, Ruosong Wang, Dingli Yu +4
Recent research shows that for training with loss, convolutional neural networks (CNNs) whose width (number of channels in convolutional layers) goes to infinity correspon…
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks
Sanjeev Arora, Simon S. Du, Zhiyuan Li +3
Recent research shows that the following two models are equivalent: (a) infinitely wide neural networks (NNs) trained under l2 loss by gradient descent with infinitesimally small l…
Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee
Wei Hu, Zhiyuan Li, Dingli Yu
Over-parameterized deep neural networks trained by simple first-order methods are known to be able to fit any labeling of data. Such over-fitting ability hinders generalization whe…
On Exact Computation with an Infinitely Wide Neural Net
Sanjeev Arora, Simon S. Du, Wei Hu +3
How well does a classic deep net architecture like AlexNet or VGG19 classify on a standard dataset such as CIFAR-10 when its width --- namely, number of channels in convolutional l…
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks
Sanjeev Arora, Simon S. Du, Wei Hu +2
Recent works have cast some light on the mystery of why deep nets fit any data and generalize despite being very overparametrized. This paper analyzes training and generalization f…