7 papers
A Provably Convergent Plug-and-Play Framework for Stochastic Bilevel Optimization
Tianshu Chu, Dachuan Xu, Wei Yao +2
Bilevel optimization has recently attracted significant attention in machine learning due to its wide range of applications and advanced hierarchical optimization capabilities. In…
Directional first order approach for a class of bilevel programs
Kuang Bai, Wei Yao, Jane J. Ye +1
In this paper, we study a class of bilevel optimization program, where the feasible set of the lower level program is independent of the upper level variable. For bilevel programs…
FAB: A First-Order AB-based Gradient Algorithm for Distributed Bilevel Optimization over Time-Varying Directed Graphs
Yaoshuai Ma, Xiao Wang, Wei Yao +1
Distributed optimization over time-varying directed graphs has shown promising performance in addressing challenges posed by complex communication constraints in real-world scenari…
SPABA: A Single-Loop and Probabilistic Stochastic Bilevel Algorithm Achieving Optimal Sample Complexity
Tianshu Chu, Dachuan Xu, Wei Yao +1
While stochastic bilevel optimization methods have been extensively studied for addressing large-scale nested optimization problems in machine learning, it remains an open question…
SUN-DSBO: A Structured Unified Framework for Nonconvex Decentralized Stochastic Bilevel Optimization
Yaoshuai Ma, Xiao Wang, Wei Yao +1
Decentralized stochastic bilevel optimization (DSBO) is a powerful tool for various machine learning tasks, including decentralized meta-learning and hyperparameter tuning. Existin…
Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization
Youran Dong, Junfeng Yang, Wei Yao +1
Bilevel optimization is a powerful tool for many machine learning problems, such as hyperparameter optimization and meta-learning. Estimating hypergradients (also known as implicit…