6 papers · 1 filter
Recursive Gradient Retraction Methods for Stochastic Nonconvex Optimization on Stiefel Manifolds
Zhijie Yuan, Lei Sun, Ganzhao Yuan
Nonconvex optimization on Stiefel manifolds underlies many problems in machine learning and deep learning. However, existing stochastic Riemannian retraction methods often rely on…
OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality
Ganzhao Yuan
Orthogonalized momentum updates, as used in Muon-style optimizers, have recently shown strong empirical stability in large-scale deep learning. However, most current orthogonalized…
Block Coordinate Descent Methods for Structured Nonconvex Optimization with Nonseparable Constraints: Optimality Conditions and Global Convergence
Zhijie Yuan, Ganzhao Yuan, Lei Sun
Coordinate descent algorithms are widely used in machine learning and large-scale data analysis due to their strong optimality guarantees and impressive empirical performance in so…
ADMM for Structured Fractional Minimization
Ganzhao Yuan
This paper considers a class of structured fractional minimization problems. The numerator consists of a differentiable function, a simple nonconvex nonsmooth function, a concave n…
ADMM for Nonsmooth Composite Optimization under Orthogonality Constraints
Ganzhao Yuan
We consider a class of structured, nonconvex, nonsmooth optimization problems under orthogonality constraints, where the objectives combine a smooth function, a nonsmooth concave f…
ADMM for Nonconvex Optimization under Minimal Continuity Assumption
Ganzhao Yuan
This paper introduces a novel approach to solving multi-block nonconvex composite optimization problems through a proximal linearized Alternating Direction Method of Multipliers (A…