3 papers
cs.LG2026
Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training
Chuan He, Zhanwang Deng, Zhaosong Lu
Neural network (NN) training is inherently a large-scale matrix optimization problem, yet the matrix structure of NN parameters has long been overlooked. Recently, the optimizer Mu…
math.OC2026
Complexity of normalized stochastic first-order methods with momentum under heavy-tailed noise
Chuan He, Zhaosong Lu, Defeng Sun +1
In this paper, we propose practical normalized stochastic first-order methods with Polyak momentum, multi-extrapolated momentum, and recursive momentum for solving unconstrained op…
math.OC2025
Stochastic interior-point methods for smooth conic optimization with applications
Chuan He, Zhanwang Deng
Conic optimization plays a crucial role in many machine learning (ML) problems. However, practical algorithms for conic constrained ML problems with large datasets are often limite…