52 citations · 289 across the 30 of their papers we have counts for
10 papers · 1 filter
A Novel Convergence Analysis for Algorithms of the Adam Family
Zhishuai Guo, Yi Xu, Wotao Yin +2
Since its invention in 2014, the Adam optimizer has received tremendous attention. On one hand, it has been widely used in deep learning and many variants have been proposed, while…
Self-Supervised Pre-Training for Transformer-Based Person Re-Identification
Hao Luo, Pichao Wang, Yi Xu +5
Transformer-based supervised pre-training achieves great performance in person re-identification (ReID). However, due to the domain gap between ImageNet and ReID datasets, it usual…
Improved Fine-Tuning by Better Leveraging Pre-Training Data
Ziquan Liu, Yi Xu, Yuanhong Xu +5
As a dominant paradigm, fine-tuning a pre-trained model on the target data is widely used in many deep learning applications, especially for small data sets. However, recent studie…
Dash: Semi-Supervised Learning with Dynamic Thresholding
Yi Xu, Lei Shang, Jinxing Ye +5
While semi-supervised learning (SSL) has received tremendous attentions in many machine learning tasks due to its successful use of unlabeled data, existing SSL algorithms use eith…
Effective Model Sparsification by Scheduled Grow-and-Prune Methods
Xiaolong Ma, Minghai Qin, Fei Sun +7
Deep neural networks (DNNs) are effective in solving many real-world problems. Larger DNN models usually exhibit better quality (e.g., accuracy) but their excessive computation res…
Why Does Multi-Epoch Training Help?
Yi Xu, Qi Qian, Hao Li +1
Stochastic gradient descent (SGD) has become the most attractive optimization method in training large-scale deep neural networks due to its simplicity, low computational cost in e…
A Theoretical Analysis of Learning with Noisily Labeled Data
Yi Xu, Qi Qian, Hao Li +1
Noisy labels are very common in deep supervised learning. Although many studies tend to improve the robustness of deep training for noisy labels, rare works focus on theoretically…
Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator
Zhishuai Guo, Yi Xu, Wotao Yin +2
Although adaptive optimization algorithms have been successful in many applications, there are still some mysteries in terms of convergence analysis that have not been unraveled. T…
A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks
Asaf Noy, Yi Xu, Yonathan Aflalo +2
Deep neural networks' remarkable ability to correctly fit training data when optimized by gradient-based algorithms is yet to be fully understood. Recent theoretical results explai…
Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity
Zhuoning Yuan, Zhishuai Guo, Yi Xu +2
Deep AUC (area under the ROC curve) Maximization (DAM) has attracted much attention recently due to its great potential for imbalanced data classification. However, the research on…