13 citations · 42 across the 24 of their papers we have counts for
26 papers
Single-Loop Gradient Algorithms for Pessimistic Bilevel Optimization Problems
Qichao Cao, Bo Zeng, Shangzhi Zeng +1
Bilevel optimization has recently attracted growing attention, particularly in the development of efficient numerical methods. Despite substantial progress on optimistic bilevel op…
Limiting Stationarity of Regularized Gap-Function Reformulations for Bilevel Optimization with Unbounded Multipliers
Xiaoning Bai, Shangzhi Zeng, Jin Zhang
Value-function-type reformulations have generated a broad class of methods for bilevel optimization. However, the corresponding value-function-type constraints are inherently degen…
A Single-Loop Penalty-based Algorithm for Stochastic Minimax Optimization with Nonlinear Coupled Constraints
Qichao Cao, Shangzhi Zeng, Jin Zhang +1
We study stochastic nonconvex-concave minimax optimization with nonlinear coupled constraints that are convex in the maximization variable. To address the nonsmoothness arising fro…
A Single-Loop Bilevel Deep Learning Method for Optimal Control of Obstacle Problems
Yongcun Song, Shangzhi Zeng, Jin Zhang +1
Optimal control of obstacle problems arises in a wide range of applications and is computationally challenging due to its nonsmoothness, nonlinearity, and bilevel structure. Classi…
A Single-Loop Gradient Algorithm for Pessimistic Bilevel Optimization via Smooth Approximation
Qichao Cao, Shangzhi Zeng, Jin Zhang
Bilevel optimization has garnered significant attention in the machine learning community recently, particularly regarding the development of efficient numerical methods. While sub…
Alternating Gradient-Type Algorithm for Bilevel Optimization with Inexact Lower-Level Solutions via Moreau Envelope-based Reformulation
Xiaoning Bai, Shangzhi Zeng, Jin Zhang +1
In this paper, we study a class of bilevel optimization problems where the lower-level problem is a convex composite optimization model, which arises in various applications, inclu…
Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-loop and Hessian-free Solution Strategy
Risheng Liu, Zhu Liu, Wei Yao +2
This work focuses on addressing two major challenges in the context of large-scale nonconvex Bi-Level Optimization (BLO) problems, which are increasingly applied in machine learnin…
Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithm
Wei Yao, Chengming Yu, Shangzhi Zeng +1
This paper presents a new approach and algorithm for solving a class of constrained Bi-Level Optimization (BLO) problems in which the lower-level problem involves constraints coupl…
Moreau Envelope Based Difference-of-weakly-Convex Reformulation and Algorithm for Bilevel Programs
Lucy L. Gao, Jane J. Ye, Haian Yin +2
Bilevel programming has emerged as a valuable tool for hyperparameter selection, a central concern in machine learning. In a recent study by Ye et al. (2023), a value function-base…
Augmenting Iterative Trajectory for Bilevel Optimization: Methodology, Analysis and Extensions
Risheng Liu, Yaohua Liu, Shangzhi Zeng +1
In recent years, there has been a surge of machine learning applications developed with hierarchical structure, which can be approached from Bi-Level Optimization (BLO) perspective…
Hierarchical Optimization-Derived Learning
Risheng Liu, Xuan Liu, Shangzhi Zeng +2
In recent years, by utilizing optimization techniques to formulate the propagation of deep model, a variety of so-called Optimization-Derived Learning (ODL) approaches have been pr…
Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexity
Risheng Liu, Yaohua Liu, Wei Yao +2
Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning fields. The validity of existing works heavily rely on either a restrictive Lower-Lev…
Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-training
Risheng Liu, Xuan Liu, Shangzhi Zeng +2
Recently, Optimization-Derived Learning (ODL) has attracted attention from learning and vision areas, which designs learning models from the perspective of optimization. However, p…
Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems
Lucy Gao, Jane J. Ye, Haian Yin +2
Gradient-based optimization methods for hyperparameter tuning guarantee theoretical convergence to stationary solutions when for fixed upper-level variable values, the lower level…
A modularized algorithmic framework for interface related optimization problems using characteristic functions
Dong Wang, Shangzhi Zeng, Jin Zhang
In this paper, we consider the algorithms and convergence for a general optimization problem, which has a wide range of applications in image segmentation, topology optimization, f…
Towards Extremely Fast Bilevel Optimization with Self-governed Convergence Guarantees
Risheng Liu, Xuan Liu, Wei Yao +2
Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning and vision fields. The validity of existing works heavily relies on solving a series…
Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond
Risheng Liu, Yaohua Liu, Shangzhi Zeng +1
In recent years, Bi-Level Optimization (BLO) techniques have received extensive attentions from both learning and vision communities. A variety of BLO models in complex and practic…
Value-Function-based Sequential Minimization for Bi-level Optimization
Risheng Liu, Xuan Liu, Shangzhi Zeng +2
Gradient-based Bi-Level Optimization (BLO) methods have been widely applied to handle modern learning tasks. However, most existing strategies are theoretically designed based on r…
A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization
Risheng Liu, Xuan Liu, Xiaoming Yuan +2
Bi-level optimization model is able to capture a wide range of complex learning tasks with practical interest. Due to the witnessed efficiency in solving bi-level programs, gradien…
A General Descent Aggregation Framework for Gradient-based Bi-level Optimization
Risheng Liu, Pan Mu, Xiaoming Yuan +2
In recent years, a variety of gradient-based methods have been developed to solve Bi-Level Optimization (BLO) problems in machine learning and computer vision areas. However, the t…
Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond
Risheng Liu, Jiaxin Gao, Jin Zhang +2
Bi-Level Optimization (BLO) is originated from the area of economic game theory and then introduced into the optimization community. BLO is able to handle problems with a hierarchi…
A Globally Convergent Proximal Newton-Type Method in Nonsmooth Convex Optimization
Boris S. Mordukhovich, Xiaoming Yuan, Shangzhi Zeng +1
The paper proposes and justifies a new algorithm of the proximal Newton type to solve a broad class of nonsmooth composite convex optimization problems without strong convexity ass…
A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton
Risheng Liu, Pan Mu, Xiaoming Yuan +2
In recent years, a variety of gradient-based first-order methods have been developed to solve bi-level optimization problems for learning applications. However, theoretical guarant…
Investigating Customization Strategies and Convergence Behaviors of Task-specific ADMM
Risheng Liu, Pan Mu, Jin Zhang
Alternating Direction Method of Multiplier (ADMM) has been a popular algorithmic framework for separable optimization problems with linear constraints. For numerical ADMM fail to e…