9 papers
Recursive Gradient Retraction Methods for Stochastic Nonconvex Optimization on Stiefel Manifolds
Zhijie Yuan, Lei Sun, Ganzhao Yuan
Nonconvex optimization on Stiefel manifolds underlies many problems in machine learning and deep learning. However, existing stochastic Riemannian retraction methods often rely on…
OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality
Ganzhao Yuan
Orthogonalized momentum updates, as used in Muon-style optimizers, have recently shown strong empirical stability in large-scale deep learning. However, most current orthogonalized…
Adaptive Lipschitz-Free Conditional Gradient Methods for Stochastic Composite Nonconvex Optimization
Ganzhao Yuan
We propose ALFCG (Adaptive Lipschitz-Free Conditional Gradient), the first \textit{adaptive} projection-free framework for stochastic composite nonconvex minimization that \textit{…
OptEMA: Adaptive Exponential Moving Average for Stochastic Optimization with Zero-Noise Optimality
Ganzhao Yuan
Exponential moving averages (EMAs) are a central component of widely used adaptive optimizers such as Adam. However, existing analyses of Adam-style methods often yield suboptimal…
Block Coordinate Descent Methods for Structured Nonconvex Optimization with Nonseparable Constraints: Optimality Conditions and Global Convergence
Zhijie Yuan, Ganzhao Yuan, Lei Sun
Coordinate descent algorithms are widely used in machine learning and large-scale data analysis due to their strong optimality guarantees and impressive empirical performance in so…
ADMM for Structured Fractional Minimization
Ganzhao Yuan
This paper considers a class of structured fractional minimization problems. The numerator consists of a differentiable function, a simple nonconvex nonsmooth function, a concave n…
Block Coordinate Descent Methods for Optimization under J-Orthogonality Constraints with Applications
Di He, Ganzhao Yuan, Xiao Wang +1
The J-orthogonal matrix, also referred to as the hyperbolic orthogonal matrix, is a class of special orthogonal matrix in hyperbolic space, notable for its advantageous properties.…
ADMM for Nonsmooth Composite Optimization under Orthogonality Constraints
Ganzhao Yuan
We consider a class of structured, nonconvex, nonsmooth optimization problems under orthogonality constraints, where the objectives combine a smooth function, a nonsmooth concave f…
ADMM for Nonconvex Optimization under Minimal Continuity Assumption
Ganzhao Yuan
This paper introduces a novel approach to solving multi-block nonconvex composite optimization problems through a proximal linearized Alternating Direction Method of Multipliers (A…