7 papers · 1 filter
UGM: A Unified Framework and New Perspectives for Accelerated Gradient Methods in Smooth and Strongly Convex Optimization
Danqing Zhou, Shiqian Ma, Junfeng Yang
In this paper, we propose a unified framework for accelerated gradient methods, dubbed UGM, which subsumes a wide range of accelerated and conventional gradient-type methods design…
SLDBO: A Snapshot Single-Loop Algorithm for Decentralized Bilevel Optimization
Chao Yin, Youran Dong, Shiqian Ma +2
Networked AI systems increasingly rely on multiple agents that collaboratively learn and adapt models over communication networks. In such systems, bilevel formulations naturally a…
On the Convergence of Constrained Gradient Method
Danqing Zhou, Hongmei Chen, Shiqian Ma +1
The constrained gradient method (CGM) has recently been proposed to solve convex optimization and monotone variational inequality (VI) problems with general functional constraints.…
A Simple Adaptive Proximal Gradient Method for Nonconvex Optimization
Zilong Ye, Shiqian Ma, Junfeng Yang +1
Consider composite nonconvex optimization problems where the objective function consists of a smooth nonconvex term (with Lipschitz-continuous gradient) and a convex (possibly nons…
Relaxed Proximal Point Algorithm: Tight Complexity Bounds and Acceleration without Momentum
Bofan Wang, Shiqian Ma, Junfeng Yang +1
In this paper, we focus on the relaxed proximal point algorithm (RPPA) for solving convex (possibly nonsmooth) optimization problems. We conduct a comprehensive study on three type…
AdaBB: Adaptive Barzilai-Borwein Method for Convex Optimization
Danqing Zhou, Shiqian Ma, Junfeng Yang
In this paper, we propose AdaBB, an adaptive gradient method based on the Barzilai-Borwein stepsize. The algorithm is line-search-free and parameter-free, and essentially provides…