Publications (19)
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel
Colin Wei, Jason D. Lee, Qiang Liu +1
Recent works have shown that on sufficiently over-parametrized neural nets, gradient descent with relatively large initialization optimizes a prediction function in the RKHS of the…
General Bounds on Satisfiability Thresholds for Random CSPs via Fourier Analysis
Colin Wei, Stefano Ermon
Random constraint satisfaction problems (CSPs) have been widely studied both in AI and complexity theory. Empirically and theoretically, many random CSPs have been shown to exhibit…
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation
Colin Wei, Tengyu Ma
Existing Rademacher complexity bounds for neural networks rely only on norm control of the weight matrices and depend exponentially on depth via a product of the matrix norms. Lowe…
Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
Colin Wei, Tengyu Ma
For linear classifiers, the relationship between (normalized) output margin and generalization is captured in a clear and simple bound -- a large output margin implies good general…
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks
Yuanzhi Li, Colin Wei, Tengyu Ma
Stochastic gradient descent with a large initial learning rate is widely used for training modern neural net architectures. Although a small initial learning rate allows for faster…
Self-training Avoids Using Spurious Features Under Domain Shift
Yining Chen, Colin Wei, Ananya Kumar +1
In unsupervised domain adaptation, existing theory focuses on situations where the source and target domains are close. In practice, conditional entropy minimization and pseudo-lab…