4 papers
When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis
Xiang Li, Zebang Shen, Ya-Ping Hsieh +1
Score-based methods, such as diffusion models and Bayesian inverse problems, are often interpreted as learning the data distribution in the low-noise limit (). In this work…
Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes
Yan Huang, Xiang Li, Yipeng Shen +2
In this paper, we show that applying adaptive methods directly to distributed minimax problems can result in non-convergence due to inconsistency in locally computed adaptive steps…
A Hessian-Aware Stochastic Differential Equation for Modelling SGD
Xiang Li, Zebang Shen, Liang Zhang +1
Continuous-time approximation of Stochastic Gradient Descent (SGD) is a crucial tool to study its escaping behaviors from stationary points. However, existing stochastic differenti…
Parameter-Agnostic Optimization under Relaxed Smoothness
Florian Hübler, Junchi Yang, Xiang Li +1
Tuning hyperparameters, such as the stepsize, presents a major challenge of training machine learning models. To address this challenge, numerous adaptive optimization algorithms h…