11 papers
A Riemannian Approach to Low-Rank Optimal Transport
Pratik Jawanpuria, Bamdev Mishra
Low-rank optimal transport (OT) mitigates the quadratic scaling of classical solvers, yet existing approaches rely heavily on first-order mirror-descent updates that require carefu…
Minibatch Selection for Language Models via Partition Matroid Constrained Gradient Matching
Prayas Agrawal, Prateek Chanda, Ishita Khatri +3
Training large language models (LLMs) on heterogeneous data requires selecting minibatches that balance convergence speed with coverage across domains. Existing methods either sele…
Riemannian Optimization for Hadamard Products of Low-Rank Matrices
Pratik Jawanpuria, Ankish Chandresh, Bamdev Mishra
The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the pr…
Nyström Approximation on Manifolds
Hantao Nie, Bin Gao, Andi Han +3
Computations on a manifold often involve constructing an operator on the tangent space and computing its inverse, which can be time-consuming in many applications. In order to redu…
LOFT: Low-Rank Orthogonal Fine-Tuning via Task-Aware Support Selection
Lanxin Zhao, Bamdev Mishra, Pratik Jawanpuria +4
Orthogonal parameter-efficient fine-tuning (PEFT) adapts pretrained weights through structure-preserving multiplicative transformations, but existing methods often conflate two dis…
Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
Yibang Li, Bihari Lal Pandey, Ravi Sah +4
Muon and related norm-constrained matrix optimizers have become central to large-scale learning problems. They are formulated as a linear maximization oracle (LMO) over an ambient…