15 papers
A Fixed-Penalty Linearized Augmented Lagrangian Method with Classical Multiplier Updates
Benqi Liu, Kangkang Deng, Zichen Wang +1
Augmented Lagrangian methods are effective for nonlinear equality-constrained optimization, but solving their nonlinear primal subproblems can be expensive. For smooth nonconvex pr…
Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning
Yuan Zhang, Jiang Hu, Zhijian Lai +2
Optimization over the Stiefel manifold plays a significant role in various machine learning tasks. Existing methods either use the retraction operators, requiring costly orthonorma…
Restarted Reflected Halpern Acceleration for Augmented Primal-Dual Methods
Benqi Liu, Ju Cao, Wotao Yin +1
We study linearly constrained composite convex optimization with a smooth term and a proximable nonsmooth term. We develop a unified augmented primal-dual framework with primal-dua…
Achieving double-logarithmic precision dependence in optimization-based quantum unstructured search
Zhijian Lai, Dong An, Jiang Hu +1
Grover's algorithm is a fundamental quantum algorithm that achieves a quadratic speedup for unstructured search problems of size . Recent studies have reformulated this task as…
A Human-AI Collaborative Workflow for Mathematical Discovery: A Case Study in Grover-Compatible Riemannian Optimization
Chenyi Li, Zhijian Lai, Dong An +2
We investigate how large language models can be used as research tools in scientific computing while preserving mathematical rigor. We propose a human-in-the-loop workflow for inte…
A Grover-compatible manifold optimization algorithm for quantum search
Zhijian Lai, Dong An, Jiang Hu +1
Grover's algorithm is a fundamental quantum algorithm that offers a quadratic speedup for the unstructured search problem by alternately applying physically implementable oracle an…
The Augmented Lagrangian Methods: Overview and Recent Advances
Kangkang Deng, Rui Wang, Zhenyuan Zhu +2
Large-scale constrained optimization is pivotal in modern scientific, engineering, and industrial computation, often involving complex systems with numerous variables and constrain…
SSNCVX: A primal-dual semismooth Newton method for convex composite optimization problem
Zhanwang Deng, Tao Wei, Jirui Ma +1
In this paper, we propose a uniform semismooth Newton-based algorithmic framework called SSNCVX for solving a broad class of convex composite optimization problems. By exploiting t…
Extended parameter shift rules with minimal derivative variance for parameterized quantum circuits
Zhijian Lai, Jiang Hu, Dong An +1
Parameter shift rules (PSRs) are useful methods for computing arbitrary-order derivatives of the cost function in parameterized quantum circuits. The basic idea of PSRs is to evalu…
Riemannian EXTRA: Communication-efficient decentralized optimization over compact submanifolds with data heterogeneity
Jiayuan Wu, Zhanwang Deng, Jiang Hu +2
We consider decentralized optimization over a compact Riemannian submanifold in a network of agents, where each agent holds a smooth, nonconvex local objective defined by its p…
LMask: Learn to Solve Constrained Routing Problems with Lazy Masking
Tianyou Li, Haijun Zou, Jiayuan Wu +1
Routing problems are canonical combinatorial optimization tasks with wide-ranging applications in logistics, transportation, and supply chain management. However, solving these pro…
An efficient primal dual semismooth Newton method for semidefinite programming
Zhanwang Deng, Jiang Hu, Kangkang Deng +1
In this paper, we present an efficient semismooth Newton method, named SSNCP, for solving a class of semidefinite programming problems. Our approach is rooted in an equivalent semi…
An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks
Zhifa Ke, Zaiwen Wen, Junyu Zhang
Temporal difference (TD) learning algorithms with neural network function parameterization have well-established empirical success in many practical large-scale reinforcement learn…
Oracle complexities of augmented Lagrangian methods for nonsmooth manifold optimization
Kangkang Deng, Jiang Hu, Jiayuan Wu +1
In this paper, we present two novel manifold inexact augmented Lagrangian methods, \textbf{ManIAL} for deterministic settings and \textbf{StoManIAL} for stochastic settings, solvin…
An Augmented Lagrangian Primal-Dual Semismooth Newton Method for Multi-Block Composite Optimization
Zhanwang Deng, Kangkang Deng, Jiang Hu +1
In this paper, we develop a novel primal-dual semismooth Newton method for solving linearly constrained multi-block convex composite optimization problems. First, a differentiable…