30 citations · 148 across the 30 of their papers we have counts for
31 papers
Stochastic Gradient Descent with Momentum is Algorithmically Stable
Yunwen Lei, Zimeng Wang, Xiaoming Yuan
Stochastic gradient descent with momentum (SGDM) is one of the most widely used optimization algorithms in machine learning. While optimization properties of SGDM have been extensi…
Learning Theory of the SVRG: Generalization and Convergence Analysis
Yunwen Lei, Zimeng Wang, Xiaoming Yuan
Variance reduction (VR) methods employ stochastic gradients with decreasing variance, and they have been widely applied to solve large-scale optimization problems in machine learni…
Towards Initialization-dependent and Non-vacuous Generalization Bounds for Overparameterized Shallow Neural Networks
Yunwen Lei, Yufeng Xie
Overparameterized neural networks often show a benign overfitting property in the sense of achieving excellent generalization behavior despite the number of parameters exceeding th…
Generalization Guarantees of Gradient Descent for Multi-Layer Neural Networks
Puyu Wang, Yunwen Lei, Di Wang +2
Recently, significant progress has been made in understanding the generalization of neural networks (NNs) trained by gradient descent (GD) using the algorithmic stability approach.…
Generalization Analysis for Contrastive Representation Learning
Yunwen Lei, Tianbao Yang, Yiming Ying +1
Recently, contrastive learning has found impressive success in advancing the state of the art in solving various machine learning tasks. However, the existing generalization analys…
Generalization Bounds for Inductive Matrix Completion in Low-noise Settings
Antoine Ledent, Rodrigo Alves, Yunwen Lei +2
We study inductive matrix completion (matrix completion with side information) under an i.i.d. subgaussian noise assumption at a low noise regime, with uniform sampling of the entr…
On Stability and Generalization of Bilevel Optimization Problem
Meng Ding, Mingxi Lei, Yunwen Lei +2
(Stochastic) bilevel optimization is a frequently encountered problem in machine learning with a wide range of applications such as meta-learning, hyper-parameter optimization, and…
Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks
Yunwen Lei, Rong Jin, Yiming Ying
While significant theoretical progress has been achieved, unveiling the generalization mystery of overparameterized neural networks still remains largely elusive. In this paper, we…
Stability and Generalization for Markov Chain Stochastic Gradient Methods
Puyu Wang, Yunwen Lei, Yiming Ying +1
Recently there is a large amount of work devoted to the study of Markov chain stochastic gradient methods (MC-SGMs) which mainly focus on their convergence analysis for solving min…
Differentially Private Stochastic Gradient Descent with Low-Noise
Puyu Wang, Yunwen Lei, Yiming Ying +1
Modern machine learning algorithms aim to extract fine-grained information from data to provide accurate predictions, which often conflicts with the goal of privacy protection. Thi…
Stability and Generalization of Stochastic Optimization with Nonconvex and Nonsmooth Problems
Yunwen Lei
Stochastic optimization has found wide applications in minimizing objective functions in machine learning, which motivates a lot of theoretical studies to understand its practical…
Differentially Private SGDA for Minimax Problems
Zhenhuan Yang, Shu Hu, Yunwen Lei +3
Stochastic gradient descent ascent (SGDA) and its variants have been the workhorse for solving minimax problems. However, in contrast to the well-studied stochastic gradient descen…
Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning
Zhenhuan Yang, Yunwen Lei, Puyu Wang +2
Pairwise learning refers to learning tasks where the loss function depends on a pair of instances. It instantiates many important machine learning tasks such as bipartite ranking a…
Stability and Generalization for Randomized Coordinate Descent
Puyu Wang, Liang Wu, Yunwen Lei
Randomized coordinate descent (RCD) is a popular optimization algorithm with wide applications in solving various machine learning problems, which motivates a lot of theoretical an…
Fine-grained Generalization Analysis of Structured Output Prediction
Waleed Mustafa, Yunwen Lei, Antoine Ledent +1
In machine learning we often encounter structured output prediction problems (SOPPs), i.e. problems where the output space admits a rich internal structure. Application domains whe…
Stability and Generalization of Stochastic Gradient Methods for Minimax Problems
Yunwen Lei, Zhenhuan Yang, Tianbao Yang +1
Many machine learning problems can be formulated as minimax problems such as Generative Adversarial Networks (GANs), AUC maximization and robust estimation, to mention but a few. A…
Fine-grained Generalization Analysis of Vector-valued Learning
Liang Wu, Antoine Ledent, Yunwen Lei +1
Many fundamental machine learning tasks can be formulated as a problem of learning with vector-valued functions, where we learn multiple scalar-valued functions together. Although…
Differentially Private SGD with Non-Smooth Losses
Puyu Wang, Yunwen Lei, Yiming Ying +1
In this paper, we are concerned with differentially private {stochastic gradient descent (SGD)} algorithms in the setting of stochastic convex optimization (SCO). Most of the exist…
Stochastic Hard Thresholding Algorithms for AUC Maximization
Zhenhuan Yang, Baojian Zhou, Yunwen Lei +1
In this paper, we aim to develop stochastic hard thresholding algorithms for the important problem of AUC maximization in imbalanced classification. The main challenge is the pairw…
Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent
Yunwen Lei, Yiming Ying
Recently there are a considerable amount of work devoted to the study of the algorithmic stability and generalization for stochastic gradient descent (SGD). However, the existing s…
On Performance Estimation in Automatic Algorithm Configuration
Shengcai Liu, Ke Tang, Yunwen Lei +1
Over the last decade, research on automated parameter tuning, often referred to as automatic algorithm configuration (AAC), has made significant progress. Although the usefulness o…
Stochastic Proximal AUC Maximization
Yunwen Lei, Yiming Ying
In this paper we consider the problem of maximizing the Area under the ROC curve (AUC) which is a widely used performance metric in imbalanced classification and anomaly detection.…
A Generalization Error Bound for Multi-class Domain Generalization
Aniket Anand Deshmukh, Yunwen Lei, Srinagesh Sharma +3
Domain generalization is the problem of assigning labels to an unlabeled data set, given several similar data sets for which labels have been provided. Despite considerable interes…
Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptions
Yunwen Lei, Ting Hu, Guiying Li +1
Stochastic gradient descent (SGD) is a popular and efficient method with wide applications in training deep neural nets and other nonconvex models. While the behavior of SGD is wel…