9 papers
A Composite Representation Theorem for Solution Mappings in Regularized Composite Optimization with Applications to Hausdorff Lipschitz Stability
Peixuan Yang, Wei Yao, Jin Zhang
We study the geometry and stability of solution mappings in regularized composite optimization, allowing multiple minimizers. We establish a composite representation that separates…
Directional first order approach for a class of bilevel programs
Kuang Bai, Wei Yao, Jane J. Ye +1
In this paper, we study a class of bilevel optimization program, where the feasible set of the lower level program is independent of the upper level variable. For bilevel programs…
SUN-DSBO: A Structured Unified Framework for Nonconvex Decentralized Stochastic Bilevel Optimization
Yaoshuai Ma, Xiao Wang, Wei Yao +1
Decentralized stochastic bilevel optimization (DSBO) is a powerful tool for various machine learning tasks, including decentralized meta-learning and hyperparameter tuning. Existin…
Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization
Youran Dong, Junfeng Yang, Wei Yao +1
Bilevel optimization is a powerful tool for many machine learning problems, such as hyperparameter optimization and meta-learning. Estimating hypergradients (also known as implicit…
A Provably Convergent Plug-and-Play Framework for Stochastic Bilevel Optimization
Tianshu Chu, Dachuan Xu, Wei Yao +2
Bilevel optimization has recently attracted significant attention in machine learning due to its wide range of applications and advanced hierarchical optimization capabilities. In…
qNBO: quasi-Newton Meets Bilevel Optimization
Sheng Fang, Yong-Jin Liu, Wei Yao +2
Bilevel optimization, addressing challenges in hierarchical learning tasks, has gained significant interest in machine learning. The practical implementation of the gradient descen…
Decomposition Method for Lipschitz Stability of General LASSO-type Problems
Chunhai Hu, Wei Yao, Jin Zhang
This paper introduces a decomposition-based method to investigate the Lipschitz stability of solution mappings for general LASSO-type problems with convex data fidelity and $\ell_1…
Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functions
Wei Yao, Haian Yin, Shangzhi Zeng +1
Constrained bilevel optimization tackles nested structures present in constrained learning tasks like constrained meta-learning, adversarial learning, and distributed bilevel optim…
SPABA: A Single-Loop and Probabilistic Stochastic Bilevel Algorithm Achieving Optimal Sample Complexity
Tianshu Chu, Dachuan Xu, Wei Yao +1
While stochastic bilevel optimization methods have been extensively studied for addressing large-scale nested optimization problems in machine learning, it remains an open question…