2 papers
cs.LG2024
SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining
Andi Han, Jiaxiang Li, Wei Huang +4
Large language models (LLMs) have shown impressive capabilities across various tasks. However, training LLMs from scratch requires significant computational power and extensive mem…
math.OC2024
A Framework for Bilevel Optimization on Riemannian Manifolds
Andi Han, Bamdev Mishra, Pratik Jawanpuria +1
Bilevel optimization has gained prominence in various applications. In this study, we introduce a framework for solving bilevel optimization problems, where the variables in both t…