5 papers
Generalized infinite dimensional Alpha-Procrustes based geometries
Salvish Goomanee, Andi Han, Pratik Jawanpuria +1
This work extends the recently introduced Alpha-Procrustes family of Riemannian metrics for symmetric positive definite (SPD) matrices by incorporating generalized versions of the…
Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method
Andi Han, Pierre-Louis Poirion, Akiko Takeda
Optimization with orthogonality constraints frequently arises in various fields such as machine learning. Riemannian optimization offers a powerful framework for solving these prob…
The Adaptive Complexity of Finding a Stationary Point
Huanjian Zhou, Andi Han, Akiko Takeda +1
In large-scale applications, such as machine learning, it is desirable to design non-convex optimization algorithms with a high degree of parallelization. In this work, we study th…
SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining
Andi Han, Jiaxiang Li, Wei Huang +4
Large language models (LLMs) have shown impressive capabilities across various tasks. However, training LLMs from scratch requires significant computational power and extensive mem…
A Framework for Bilevel Optimization on Riemannian Manifolds
Andi Han, Bamdev Mishra, Pratik Jawanpuria +1
Bilevel optimization has gained prominence in various applications. In this study, we introduce a framework for solving bilevel optimization problems, where the variables in both t…