3 papers
cs.LG2026
Asynchronous Heavy-Tailed Optimization
Junfei Sun, Dixi Yao, Xuchen Gong +3
Heavy-tailed stochastic gradient noise, commonly observed in transformer models, can destabilize the optimization process. Recent works mainly focus on developing and understanding…
cs.LG2026
Private Zeroth-Order Optimization with Public Data
Xuchen Gong, Tian Li
One of the major bottlenecks for deploying popular first-order differentially private (DP) machine learning algorithms (e.g., DP-SGD) lies in their high computation and memory cost…
cs.LG2025
Zeroth-Order Sharpness-Aware Learning with Exponential Tilting
Xuchen Gong, Tian Li
Classic zeroth-order optimization approaches typically optimize for a smoothed version of the original function, i.e., the expected objective under randomly perturbed model paramet…