9 papers · 1 filter
Variational analysis of determinantal varieties
Yan Yang, Bin Gao, Ya-xiang Yuan
Determinantal varieties -- the sets of bounded-rank matrices or tensors -- have attracted growing interest in low-rank optimization. The tangent cone to low-rank sets is widely stu…
Distributed Retraction-Free and Communication-Efficient Optimization on the Stiefel Manifold
Yilong Song, Peijin Li, Bin Gao +1
Optimization problems on the Stiefel manifold, ranging from principal component analysis to enhancing neural network robustness, are ubiquitous in machine learning. The Landing alg…
First-order methods on bounded-rank tensors converging to stationary points
Bin Gao, Renfeng Peng, Ya-xiang Yuan
Provably finding stationary points on bounded-rank tensors turns out to be an open problem [E. Levin, J. Kileel, and N. Boumal, Math. Program., 199 (2023), pp. 831--864] due to the…
A space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints
Yan Yang, Bin Gao, Ya-xiang Yuan
Imposing additional constraints on low-rank optimization has garnered growing interest. However, the geometry of coupled constraints hampers the well-developed low-rank structure a…
Desingularization of bounded-rank tensor sets
Bin Gao, Renfeng Peng, Ya-xiang Yuan
Low-rank tensors appear to be prosperous in many applications. However, the sets of bounded-rank tensors are non-smooth and non-convex algebraic varieties, rendering the low-rank o…
Symplectic Stiefel manifold: tractable metrics, second-order geometry and Newton's methods
Bin Gao, Nguyen Thanh Son, Tatjana Stykel
Optimization under the symplecticity constraint is an approach for solving various problems in quantum physics and scientific computing. Building on the results that this optimizat…
Bilevel reinforcement learning via the development of hyper-gradient without lower-level convexity
Yan Yang, Bin Gao, Ya-xiang Yuan
Bilevel reinforcement learning (RL), which features intertwined two-level problems, has attracted growing interest recently. The inherent non-convexity of the lower-level RL proble…
LancBiO: dynamic Lanczos-aided bilevel optimization via Krylov subspace
Yan Yang, Bin Gao, Ya-xiang Yuan
Bilevel optimization, with broad applications in machine learning, has an intricate hierarchical structure. Gradient-based methods have emerged as a common approach to large-scale…
Low-rank optimization on Tucker tensor varieties
Bin Gao, Renfeng Peng, Ya-xiang Yuan
In the realm of tensor optimization, the low-rank Tucker decomposition is crucial for reducing the number of parameters and for saving storage. We explore the geometry of Tucker te…