52 citations · 289 across the 30 of their papers we have counts for
30 papers
Fairness via Adversarial Attribute Neighbourhood Robust Learning
Qi Qi, Shervin Ardeshir, Yi Xu +1
Improving fairness between privileged and less-privileged sensitive attribute groups (e.g, {race, gender}) has attracted lots of attention. To enhance the model performs uniformly…
An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation
Ziquan Liu, Yi Xu, Yuanhong Xu +5
The performance of machine learning models under distribution shift has been the focus of the community in recent years. Most of current methods have been proposed to improve the r…
A Novel Convergence Analysis for Algorithms of the Adam Family
Zhishuai Guo, Yi Xu, Wotao Yin +2
Since its invention in 2014, the Adam optimizer has received tremendous attention. On one hand, it has been widely used in deep learning and many variants have been proposed, while…
Self-Supervised Pre-Training for Transformer-Based Person Re-Identification
Hao Luo, Pichao Wang, Yi Xu +5
Transformer-based supervised pre-training achieves great performance in person re-identification (ReID). However, due to the domain gap between ImageNet and ReID datasets, it usual…
Improved Fine-Tuning by Better Leveraging Pre-Training Data
Ziquan Liu, Yi Xu, Yuanhong Xu +5
As a dominant paradigm, fine-tuning a pre-trained model on the target data is widely used in many deep learning applications, especially for small data sets. However, recent studie…
Dash: Semi-Supervised Learning with Dynamic Thresholding
Yi Xu, Lei Shang, Jinxing Ye +5
While semi-supervised learning (SSL) has received tremendous attentions in many machine learning tasks due to its successful use of unlabeled data, existing SSL algorithms use eith…
Effective Model Sparsification by Scheduled Grow-and-Prune Methods
Xiaolong Ma, Minghai Qin, Fei Sun +7
Deep neural networks (DNNs) are effective in solving many real-world problems. Larger DNN models usually exhibit better quality (e.g., accuracy) but their excessive computation res…
Why Does Multi-Epoch Training Help?
Yi Xu, Qi Qian, Hao Li +1
Stochastic gradient descent (SGD) has become the most attractive optimization method in training large-scale deep neural networks due to its simplicity, low computational cost in e…
A Theoretical Analysis of Learning with Noisily Labeled Data
Yi Xu, Qi Qian, Hao Li +1
Noisy labels are very common in deep supervised learning. Although many studies tend to improve the robustness of deep training for noisy labels, rare works focus on theoretically…
Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator
Zhishuai Guo, Yi Xu, Wotao Yin +2
Although adaptive optimization algorithms have been successful in many applications, there are still some mysteries in terms of convergence analysis that have not been unraveled. T…
A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks
Asaf Noy, Yi Xu, Yonathan Aflalo +2
Deep neural networks' remarkable ability to correctly fit training data when optimized by gradient-based algorithms is yet to be fully understood. Recent theoretical results explai…
Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity
Zhuoning Yuan, Zhishuai Guo, Yi Xu +2
Deep AUC (area under the ROC curve) Maximization (DAM) has attracted much attention recently due to its great potential for imbalanced data classification. However, the research on…
Attentional-Biased Stochastic Gradient Descent
Qi Qi, Yi Xu, Rong Jin +2
In this paper, we present a simple yet effective provable method (named ABSGD) for addressing the data imbalance or label noise problem in deep learning. Our method is a simple mod…
WeMix: How to Better Utilize Data Augmentation
Yi Xu, Asaf Noy, Ming Lin +3
Data augmentation is a widely used training trick in deep learning to improve the network generalization ability. Despite many encouraging results, several recent studies did point…
Towards Understanding Label Smoothing
Yi Xu, Yuanhong Xu, Qi Qian +2
Label smoothing regularization (LSR) has a great success in training deep neural networks by stochastic algorithms such as stochastic gradient descent and its variants. However, th…
An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectives
Qi Qi, Zhishuai Guo, Yi Xu +2
In this paper, we propose a practical online method for solving a class of distributionally robust optimization (DRO) with non-convex objectives, which has important applications i…
Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max Optimization
Yan Yan, Yi Xu, Qihang Lin +2
Epoch gradient descent method (a.k.a. Epoch-GD) proposed by Hazan and Kale (2011) was deemed a breakthrough for stochastic strongly convex minimization, which achieves the optimal…
Stochastic Optimization for Non-convex Inf-Projection Problems
Yan Yan, Yi Xu, Lijun Zhang +2
In this paper, we study a family of non-convex and possibly non-smooth inf-projection minimization problems, where the target objective function is equal to minimization of a joint…
Stochastic Primal-Dual Algorithms with Faster Convergence than for Problems without Bilinear Structure
Yan Yan, Yi Xu, Qihang Lin +2
Previous studies on stochastic primal-dual algorithms for solving min-max problems with faster convergence heavily rely on the bilinear structure of the problem, which restricts th…
On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Optimization
Yi Xu, Zhuoning Yuan, Sen Yang +2
Extrapolation is a well-known technique for solving convex optimization and variational inequalities and recently attracts some attention for non-convex optimization. Several recen…
Non-asymptotic Analysis of Stochastic Methods for Non-Smooth Non-Convex Regularized Problems
Yi Xu, Rong Jin, Tianbao Yang
Stochastic Proximal Gradient (SPG) methods have been widely used for solving optimization problems with a simple (possibly non-smooth) regularizer in machine learning and statistic…
Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence
Yi Xu, Qi Qi, Qihang Lin +2
Difference of convex (DC) functions cover a broad family of non-convex and possibly non-smooth and non-differentiable functions, and have wide applications in machine learning and…
Frank-Wolfe Method is Automatically Adaptive to Error Bound Condition
Yi Xu, Tianbao Yang
Error bound condition has recently gained revived interest in optimization. It has been leveraged to derive faster convergence for many popular algorithms, including subgradient me…
Katalyst: Boosting Convex Katayusha for Non-Convex Problems with a Large Condition Number
Zaiyi Chen, Yi Xu, Haoyuan Hu +1
In this paper, we propose a new SVRG-style acceleated stochastic algorithm for solving a family of non-convex optimization problems whose objective consists of a sum of smooth…