1 citations · 2 across the 6 of their papers we have counts for
4 papers · 1 filter
The Convergence Behavior of Adam under Heavy-Tailed Noise
Yijiang Pang
We establish the first convergence guarantees for the plain vector-form Adam optimizer under heavy-tailed stochastic noise. While several Adam variants are known to achieve optimal…
Distributed Harmonization: Federated Clustered Batch Effect Adjustment and Generalization
Bao Hoang, Yijiang Pang, Siqi Liang +3
Independent and identically distributed (i.i.d.) data is essential to many data analysis and modeling techniques. In the medical domain, collecting data from multiple sites or inst…
Towards Stability of Parameter-Free Optimization
Yijiang Pang, Shuyang Yu, Bao Hoang +1
Hyperparameter tuning, particularly the selection of an appropriate learning rate in adaptive gradient training methods, remains a challenge. To address this challenge, we propose…
Stochastic Two Points Method for Deep Model Zeroth-order Optimization
Yijiang Pang, Jiayu Zhou
Large foundation models, such as large language models, have performed exceptionally well in various application scenarios. Building or fully fine-tuning such large models is usual…