9 papers · 1 filter
Stochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework
Junwen Qiu, Bohao Ma, Andre Milzarek +1
Unit excitation (UE) is a common assumption in stochastic saddle avoidance: the stochastic error must have a uniformly positive component along every direction, in expectation. Thi…
Schattor: Schatten-family methods for deep learning optimization
Bohao Ma, Junyu Zhang, Chuan He
Modern deep learning optimization features heterogeneous parameter structures, noisy gradients, and highly nonconvex landscapes, posing significant challenges for both algorithm de…
Adaptive Newton-CG methods with global and local analysis for unconstrained optimization with Hölder continuous Hessian
Ziyang Zeng, Junyu Zhang, Chuan He
In this paper, we study Newton-conjugate gradient (Newton-CG) methods for minimizing a nonconvex function whose Hessian is -Hölder continuous with modulus an…
Shuffling the Stochastic Mirror Descent via Dual Lipschitz Continuity and Kernel Conditioning
Junwen Qiu, Leilei Mei, Junyu Zhang
The global Lipschitz smoothness condition underlies most convergence and complexity analyses via two key consequences: the descent lemma and the gradient Lipschitz continuity. How…
A New Kernel Regularity Condition for Distributed Mirror Descent: Broader Coverage and Simpler Analysis
Junwen Qiu, Ziyang Zeng, Leilei Mei +1
Existing convergence of distributed optimization methods in non-Euclidean geometries typically rely on kernel assumptions: (i) global Lipschitz smoothness and (ii) bi-convexity of…
Line-search and Adaptive Step Sizes for Nonconvex-strongly-concave Minimax Optimization
Bohao Ma, Nachuan Xiao, Junyu Zhang
In this paper, we propose a novel reformulation of the smooth nonconvex-strongly-concave (NC-SC) minimax problems that casts the problem as a joint minimization. We show that our r…