5 papers · 1 filter
Approximation in Contrastive Representation Learning
Yuanfan Li, Zihan Zhang, Yiming Ying +1
Contrastive representation learning (CRL) is a fundamental paradigm for learning transferable representations. Recent work identifies its population target as the pointwise mutual…
Stability and Generalization of Straight-Through Estimators for Training Two-Layer Quantized Neural Networks
Yiming Ying
We study the identity straight-through estimator (STE) for training a two-layer binary-activation network with hinge loss from the perspective of Statistical Learning Theory (SLT).…
Statistical Consistency and Generalization of Contrastive Representation Learning
Yuanfan Li, Xiyuan Wei, Tianbao Yang +1
Contrastive representation learning (CRL) underpins many modern foundation models. Despite recent theoretical progress, existing analyses suffer from several key limitations: (i) t…
Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification
Yuanfan Li, Yunwen Lei, Zheng-Chu Guo +1
Recent advances have significantly improved our understanding of the generalization performance of gradient descent (GD) methods in deep neural networks. A natural and fundamental…
Differentially Private Non-convex Learning for Multi-layer Neural Networks
Hanpu Shen, Cheng-Long Wang, Zihang Xiang +2
This paper focuses on the problem of Differentially Private Stochastic Optimization for (multi-layer) fully connected neural networks with a single output node. In the first part,…