6 papers · 1 filter
Generalization Analysis of Biased Stochastic Gradient Methods for Minimax Problems
Shuang Zeng, Yunwen Lei, Yiming Ying
Minimax optimization refers to a class of optimization problems with some variables to minimize and other variables to maximize. Biased stochastic gradient methods have shown pract…
Approximation in Contrastive Representation Learning
Yuanfan Li, Zihan Zhang, Yiming Ying +1
Contrastive representation learning (CRL) is a fundamental paradigm for learning transferable representations. Recent work identifies its population target as the pointwise mutual…
Stability and Generalization of Straight-Through Estimators for Training Two-Layer Quantized Neural Networks
Yiming Ying
We study the identity straight-through estimator (STE) for training a two-layer binary-activation network with hinge loss from the perspective of Statistical Learning Theory (SLT).…
Minimax-Optimal Generalization Bounds for Smooth Deep Neural Networks Trained by (Stochastic) Gradient Descent
Junyu Zhou, Puyu Wang, Dennis Wagner +3
Characterizing the optimization dynamics and statistical performance of over-parameterized deep neural networks (DNNs) remains a central challenge in understanding the remarkable s…
Optimal Rates for Generalization of Gradient Descent Methods with Deep Neural Networks
Junyu Zhou, Puyu Wang, Yunwen Lei +2
Recent progress has been made in understanding the statistical generalization performance of gradient descent methods for overparameterized neural networks within the neural tangen…
Statistical Consistency and Generalization of Contrastive Representation Learning
Yuanfan Li, Xiyuan Wei, Tianbao Yang +1
Contrastive representation learning (CRL) underpins many modern foundation models. Despite recent theoretical progress, existing analyses suffer from several key limitations: (i) t…