7 papers
Decentralized Bilevel Optimization: A Perspective from Transient Iteration Complexity
Boao Kong, Shuchen Zhu, Songtao Lu +2
Stochastic bilevel optimization (SBO) is becoming increasingly essential in machine learning due to its versatility in handling nested structures. To address large-scale SBO, decen…
SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization
Shuchen Zhu, Boao Kong, Songtao Lu +2
This paper studies decentralized bilevel optimization, in which multiple agents collaborate to solve problems involving nested optimization structures with neighborhood communicati…
Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity
Xinmeng Huang, Kan Xu, Donghwan Lee +3
Large and complex datasets are often collected from several, possibly heterogeneous sources. Multitask learning methods improve efficiency by leveraging commonalities across datase…
Scaled Relative Graph of Normal Matrices
Xinmeng Huang, Ernest K. Ryu, Wotao Yin
The Scaled Relative Graph (SRG) is a geometric tool that maps the action of a multi-valued nonlinear operator onto the 2D plane, used to analyze the convergence of a wide range of…
A Mathematics-Inspired Learning-to-Optimize Framework for Decentralized Optimization
Yutong He, Qiulin Shang, Xinmeng Huang +2
Most decentralized optimization algorithms are handcrafted. While endowed with strong theoretical guarantees, these algorithms generally target a broad class of problems, thereby n…
Distributed Bilevel Optimization with Communication Compression
Yutong He, Jie Hu, Xinmeng Huang +3
Stochastic bilevel optimization tackles challenges involving nested optimization structures. Its fast-growing scale nowadays necessitates efficient distributed algorithms. In conve…