164 citations · 203 across the 6 of their papers we have counts for
Showing 2019Show all
3 papers · 1 filter
cs.LG2019
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks
Yuanzhi Li, Colin Wei, Tengyu Ma
Stochastic gradient descent with a large initial learning rate is widely used for training modern neural net architectures. Although a small initial learning rate allows for faster…
cs.LG2019
Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
Kaidi Cao, Colin Wei, Adrien Gaidon +2
Deep learning algorithms can fare poorly when the training dataset suffers from heavy class-imbalance but the testing criterion requires good generalization on less frequent classe…
cs.LG2019
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation
Colin Wei, Tengyu Ma
Existing Rademacher complexity bounds for neural networks rely only on norm control of the weight matrices and depend exponentially on depth via a product of the matrix norms. Lowe…