6 papers · 1 filter
SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining
Andi Han, Jiaxiang Li, Wei Huang +4
Large language models (LLMs) have shown impressive capabilities across various tasks. However, training LLMs from scratch requires significant computational power and extensive mem…
A Framework for Bilevel Optimization on Riemannian Manifolds
Andi Han, Bamdev Mishra, Pratik Jawanpuria +1
Bilevel optimization has gained prominence in various applications. In this study, we introduce a framework for solving bilevel optimization problems, where the variables in both t…
A Riemannian Approach to Ground Metric Learning for Optimal Transport
Pratik Jawanpuria, Dai Shi, Bamdev Mishra +1
Optimal transport (OT) theory has attracted much attention in machine learning and signal processing applications. OT defines a notion of distance between probability distributions…
Riemannian coordinate descent algorithms on matrix manifolds
Andi Han, Pratik Jawanpuria, Bamdev Mishra
Many machine learning applications are naturally formulated as optimization problems on Riemannian manifolds. The main idea behind Riemannian optimization is to maintain the feasib…
Federated Learning on Riemannian Manifolds with Differential Privacy
Zhenwei Huang, Wen Huang, Pratik Jawanpuria +1
In recent years, federated learning (FL) has emerged as a prominent paradigm in distributed machine learning. Despite the partial safeguarding of agents' information within FL syst…
A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks
Neel Mishra, Bamdev Mishra, Pratik Jawanpuria +1
A novel first-order method is proposed for training generative adversarial networks (GANs). It modifies the Gauss-Newton method to approximate the min-max Hessian and uses the Sher…