252 citations · 3.3k across the 92 of their papers we have counts for
9 papers · 1 filter
Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
Colin Wei, Tengyu Ma
For linear classifiers, the relationship between (normalized) output margin and generalization is captured in a clear and simple bound -- a large output margin implies good general…
Verified Uncertainty Calibration
Ananya Kumar, Percy Liang, Tengyu Ma
Applications such as weather forecasting and personalized medicine demand models that output calibrated probability estimates---those representative of the true likelihood of a pre…
Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Sampling
Yuping Luo, Huazhe Xu, Tengyu Ma
Imitation learning, followed by reinforcement learning algorithms, is a promising paradigm to solve complex control tasks sample-efficiently. However, learning from demonstrations…
A Model-based Approach for Sample-efficient Multi-task Reinforcement Learning
Nicholas C. Landolfi, Garrett Thomas, Tengyu Ma
The aim of multi-task reinforcement learning is two-fold: (1) efficiently learn by training against multiple tasks and (2) quickly adapt, using limited samples, to a variety of new…
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks
Yuanzhi Li, Colin Wei, Tengyu Ma
Stochastic gradient descent with a large initial learning rate is widely used for training modern neural net architectures. Although a small initial learning rate allows for faster…
Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
Kaidi Cao, Colin Wei, Adrien Gaidon +2
Deep learning algorithms can fare poorly when the training dataset suffers from heavy class-imbalance but the testing criterion requires good generalization on less frequent classe…
On the Performance of Thompson Sampling on Logistic Bandits
Shi Dong, Tengyu Ma, Benjamin Van Roy
We study the logistic bandit, in which rewards are binary with success probability and actions and coefficients are within the …
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation
Colin Wei, Tengyu Ma
Existing Rademacher complexity bounds for neural networks rely only on norm control of the weight matrices and depend exponentially on depth via a product of the matrix norms. Lowe…
Fixup Initialization: Residual Learning Without Normalization
Hongyi Zhang, Yann N. Dauphin, Tengyu Ma
Normalization layers are a staple in state-of-the-art deep neural network architectures. They are widely believed to stabilize training, enable higher learning rate, accelerate con…