9 papers · 1 filter
A Riemannian Approach to Low-Rank Optimal Transport
Pratik Jawanpuria, Bamdev Mishra
Low-rank optimal transport (OT) mitigates the quadratic scaling of classical solvers, yet existing approaches rely heavily on first-order mirror-descent updates that require carefu…
Minibatch Selection for Language Models via Partition Matroid Constrained Gradient Matching
Prayas Agrawal, Prateek Chanda, Ishita Khatri +3
Training large language models (LLMs) on heterogeneous data requires selecting minibatches that balance convergence speed with coverage across domains. Existing methods either sele…
Riemannian Optimization for Hadamard Products of Low-Rank Matrices
Pratik Jawanpuria, Ankish Chandresh, Bamdev Mishra
The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the pr…
LOFT: Low-Rank Orthogonal Fine-Tuning via Task-Aware Support Selection
Lanxin Zhao, Bamdev Mishra, Pratik Jawanpuria +4
Orthogonal parameter-efficient fine-tuning (PEFT) adapts pretrained weights through structure-preserving multiplicative transformations, but existing methods often conflate two dis…
Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
Yibang Li, Bihari Lal Pandey, Ravi Sah +4
Muon and related norm-constrained matrix optimizers have become central to large-scale learning problems. They are formulated as a linear maximization oracle (LMO) over an ambient…
UniPROT: Uniform Prototype Selection via Partial Optimal Transport with Submodular Guarantees
Prateek Chanda, Prayas Agrawal, Karthik S. Gurumoorthy +3
Selecting prototypical examples from a source distribution to represent a target data distribution is a fundamental problem in machine learning. Existing subset selection methods o…