13 citations · 49 across the 26 of their papers we have counts for
30 papers
A Proximal Newton Framework with Local Superlinear Convergence for Simple Bilevel Optimization
Xiaoning Bai, Boris S. Mordukhovich, Shangzhi Zeng +1
In this paper, we study a penalty method for simple bilevel optimization in which the lower-level objective has a convex composite structure. Our approach is based on a vanishing-p…
Limiting Stationarity of Regularized Gap-Function Reformulations for Bilevel Optimization with Unbounded Multipliers
Xiaoning Bai, Shangzhi Zeng, Jin Zhang
Value-function-type reformulations have generated a broad class of methods for bilevel optimization. However, the corresponding value-function-type constraints are inherently degen…
A Single-Loop Penalty-based Algorithm for Stochastic Minimax Optimization with Nonlinear Coupled Constraints
Qichao Cao, Shangzhi Zeng, Jin Zhang +1
We study stochastic nonconvex-concave minimax optimization with nonlinear coupled constraints that are convex in the maximization variable. To address the nonsmoothness arising fro…
A Single-Loop Bilevel Deep Learning Method for Optimal Control of Obstacle Problems
Yongcun Song, Shangzhi Zeng, Jin Zhang +1
Optimal control of obstacle problems arises in a wide range of applications and is computationally challenging due to its nonsmoothness, nonlinearity, and bilevel structure. Classi…
A Single-Loop Gradient Algorithm for Pessimistic Bilevel Optimization via Smooth Approximation
Qichao Cao, Shangzhi Zeng, Jin Zhang
Bilevel optimization has garnered significant attention in the machine learning community recently, particularly regarding the development of efficient numerical methods. While sub…
Prox-PINNs: A Deep Learning Algorithmic Framework for Elliptic Variational Inequalities
Yu Gao, Yongcun Song, Zhiyu Tan +2
Elliptic variational inequalities (EVIs) present significant challenges in numerical computation due to their inherent non-smoothness, nonlinearity, and inequality formulations. Tr…
Extended SQP Methods in Nonsmooth Difference Programming Applied to Problems with Variational Inequality Constraints
Boris S. Mordukhovich, Yixia Song, Shangzhi Zeng +1
This paper explores a new class of constrained difference programming problems, where the objective and constraints are formulated as differences of functions, without requiring th…
Alternating Gradient-Type Algorithm for Bilevel Optimization with Inexact Lower-Level Solutions via Moreau Envelope-based Reformulation
Xiaoning Bai, Shangzhi Zeng, Jin Zhang +1
In this paper, we study a class of bilevel optimization problems where the lower-level problem is a convex composite optimization model, which arises in various applications, inclu…
Gradient-based algorithms for multi-objective bi-level optimization
Xinmin Yang, Wei Yao, Haian Yin +2
Multi-Objective Bi-Level Optimization (MOBLO) addresses nested multi-objective optimization problems common in a range of applications. However, its multi-objective and hierarchica…
Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functions
Wei Yao, Haian Yin, Shangzhi Zeng +1
Constrained bilevel optimization tackles nested structures present in constrained learning tasks like constrained meta-learning, adversarial learning, and distributed bilevel optim…
Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-loop and Hessian-free Solution Strategy
Risheng Liu, Zhu Liu, Wei Yao +2
This work focuses on addressing two major challenges in the context of large-scale nonconvex Bi-Level Optimization (BLO) problems, which are increasingly applied in machine learnin…
Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithm
Wei Yao, Chengming Yu, Shangzhi Zeng +1
This paper presents a new approach and algorithm for solving a class of constrained Bi-Level Optimization (BLO) problems in which the lower-level problem involves constraints coupl…
Optimality conditions for bilevel programs via Moreau envelope reformulation
Kuang Bai, Jane Ye, Shangzhi Zeng
For bilevel programs with a convex lower level program, the classical approach replaces the lower level program with its Karush-Kuhn-Tucker condition and solve the resulting mathem…
Moreau Envelope Based Difference-of-weakly-Convex Reformulation and Algorithm for Bilevel Programs
Lucy L. Gao, Jane J. Ye, Haian Yin +2
Bilevel programming has emerged as a valuable tool for hyperparameter selection, a central concern in machine learning. In a recent study by Ye et al. (2023), a value function-base…
Augmenting Iterative Trajectory for Bilevel Optimization: Methodology, Analysis and Extensions
Risheng Liu, Yaohua Liu, Shangzhi Zeng +1
In recent years, there has been a surge of machine learning applications developed with hierarchical structure, which can be approached from Bi-Level Optimization (BLO) perspective…
Hierarchical Optimization-Derived Learning
Risheng Liu, Xuan Liu, Shangzhi Zeng +2
In recent years, by utilizing optimization techniques to formulate the propagation of deep model, a variety of so-called Optimization-Derived Learning (ODL) approaches have been pr…
Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexity
Risheng Liu, Yaohua Liu, Wei Yao +2
Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning fields. The validity of existing works heavily rely on either a restrictive Lower-Lev…
Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-training
Risheng Liu, Xuan Liu, Shangzhi Zeng +2
Recently, Optimization-Derived Learning (ODL) has attracted attention from learning and vision areas, which designs learning models from the perspective of optimization. However, p…
Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems
Lucy Gao, Jane J. Ye, Haian Yin +2
Gradient-based optimization methods for hyperparameter tuning guarantee theoretical convergence to stationary solutions when for fixed upper-level variable values, the lower level…
A modularized algorithmic framework for interface related optimization problems using characteristic functions
Dong Wang, Shangzhi Zeng, Jin Zhang
In this paper, we consider the algorithms and convergence for a general optimization problem, which has a wide range of applications in image segmentation, topology optimization, f…
Towards Extremely Fast Bilevel Optimization with Self-governed Convergence Guarantees
Risheng Liu, Xuan Liu, Wei Yao +2
Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning and vision fields. The validity of existing works heavily relies on solving a series…
Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond
Risheng Liu, Yaohua Liu, Shangzhi Zeng +1
In recent years, Bi-Level Optimization (BLO) techniques have received extensive attentions from both learning and vision communities. A variety of BLO models in complex and practic…
Value-Function-based Sequential Minimization for Bi-level Optimization
Risheng Liu, Xuan Liu, Shangzhi Zeng +2
Gradient-based Bi-Level Optimization (BLO) methods have been widely applied to handle modern learning tasks. However, most existing strategies are theoretically designed based on r…
A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization
Risheng Liu, Xuan Liu, Xiaoming Yuan +2
Bi-level optimization model is able to capture a wide range of complex learning tasks with practical interest. Due to the witnessed efficiency in solving bi-level programs, gradien…