8 papers
OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality
Ganzhao Yuan
Orthogonalized momentum updates, as used in Muon-style optimizers, have recently shown strong empirical stability in large-scale deep learning. However, most current orthogonalized…
OptEMA: Adaptive Exponential Moving Average for Stochastic Optimization with Zero-Noise Optimality
Ganzhao Yuan
Exponential moving averages (EMAs) are a central component of widely used adaptive optimizers such as Adam. However, existing analyses of Adam-style methods often yield suboptimal…
Adaptive Lipschitz-Free Conditional Gradient Methods for Stochastic Composite Nonconvex Optimization
Ganzhao Yuan
We propose ALFCG (Adaptive Lipschitz-Free Conditional Gradient), the first \textit{adaptive} projection-free framework for stochastic composite nonconvex minimization that \textit{…
ADMM for Structured Fractional Minimization
Ganzhao Yuan
This paper considers a class of structured fractional minimization problems. The numerator consists of a differentiable function, a simple nonconvex nonsmooth function, a concave n…
ADMM for Nonconvex Optimization under Minimal Continuity Assumption
Ganzhao Yuan
This paper introduces a novel approach to solving multi-block nonconvex composite optimization problems through a proximal linearized Alternating Direction Method of Multipliers (A…
ADMM for Nonsmooth Composite Optimization under Orthogonality Constraints
Ganzhao Yuan
We consider a class of structured, nonconvex, nonsmooth optimization problems under orthogonality constraints, where the objectives combine a smooth function, a nonsmooth concave f…