15 papers
Stochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework
Junwen Qiu, Bohao Ma, Andre Milzarek +1
Unit excitation (UE) is a common assumption in stochastic saddle avoidance: the stochastic error must have a uniformly positive component along every direction, in expectation. Thi…
Schattor: Schatten-family methods for deep learning optimization
Bohao Ma, Junyu Zhang, Chuan He
Modern deep learning optimization features heterogeneous parameter structures, noisy gradients, and highly nonconvex landscapes, posing significant challenges for both algorithm de…
Global Policy-Space Response Oracles for Two-Player Zero-Sum Games
Junyu Zhang, Feihong Yang, Jian Wang +2
The Policy-Space Response Oracles (PSRO) framework scales equilibrium computation to large zero-sum games by iteratively expanding a restricted strategy set using deep reinforcemen…
Automated Reformulation of Robust Optimization via Memory-Augmented Large Language Models
Jinbiao Chen, Shuang Jin, Guoyun Zhang +3
Robust optimization (RO) provides a principled framework for decision-making under uncertainty, but its practical use is often limited by the need to manually reformulate uncertain…
Adaptive Newton-CG methods with global and local analysis for unconstrained optimization with Hölder continuous Hessian
Ziyang Zeng, Junyu Zhang, Chuan He
In this paper, we study Newton-conjugate gradient (Newton-CG) methods for minimizing a nonconvex function whose Hessian is -Hölder continuous with modulus and…
Shuffling the Stochastic Mirror Descent via Dual Lipschitz Continuity and Kernel Conditioning
Junwen Qiu, Leilei Mei, Junyu Zhang
The global Lipschitz smoothness condition underlies most convergence and complexity analyses via two key consequences: the descent lemma and the gradient Lipschitz continuity. How…
A New Kernel Regularity Condition for Distributed Mirror Descent: Broader Coverage and Simpler Analysis
Junwen Qiu, Ziyang Zeng, Leilei Mei +1
Existing convergence analyses of distributed optimization methods in non-Euclidean geometries typically rely on kernel assumptions: (i) global Lipschitz smoothness and (ii) bi-conv…
Line-search and Adaptive Step Sizes for Nonconvex-strongly-concave Minimax Optimization
Bohao Ma, Nachuan Xiao, Junyu Zhang
In this paper, we propose a novel reformulation of the smooth nonconvex-strongly-concave (NC-SC) minimax problems that casts the problem as a joint minimization. We show that our r…
Non-Asymptotic Global Convergence of PPO-Clip
Yin Liu, Qiming Dai, Junyu Zhang +1
Reinforcement learning has gained attention for modern Large Language Model post-training. The actor-only variants of Proximal Policy Optimization (PPO) are widely applied for thei…
The Augmented Lagrangian Methods: Overview and Recent Advances
Kangkang Deng, Rui Wang, Zhenyuan Zhu +2
Large-scale constrained optimization is pivotal in modern scientific, engineering, and industrial computation, often involving complex systems with numerous variables and constrain…
Bridging the Resource Gap: Deploying Advanced Imitation Learning Models onto Affordable Embedded Platforms
Haizhou Ge, Ruixiang Wang, Zhu-ang Xu +7
Advanced imitation learning with structures like the transformer is increasingly demonstrating its advantages in robotics. However, deploying these large-scale models on embedded p…
General Procedure to Provide High-Probability Guarantees for Stochastic Saddle Point Problems
Dongyang Li, Haobin Li, Junyu Zhang
This paper considers smooth strongly convex and strongly concave (SC-SC) stochastic saddle point (SSP) problems. Suppose there is an arbitrary oracle that in expectation returns an…
A Payoff-Based Policy Gradient Method in Stochastic Games with Long-Run Average Payoffs
Junyue Zhang, Yifen Mu
Despite the significant potential for various applications, stochastic games with long-run average payoffs have received limited scholarly attention, particularly concerning the de…
An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks
Zhifa Ke, Zaiwen Wen, Junyu Zhang
Temporal difference (TD) learning algorithms with neural network function parameterization have well-established empirical success in many practical large-scale reinforcement learn…
Stochastic Bregman Proximal Gradient Method Revisited: Kernel Conditioning and Painless Variance Reduction
Junyu Zhang
We investigate stochastic Bregman proximal gradient (SBPG) methods for minimizing a finite-sum nonconvex function , where is convex an…