Minibatch Selection for Language Models via Partition Matroid Constrained Gradient Matching

Prayas Agrawal, Prateek Chanda, Ishita Khatri +3

Training large language models (LLMs) on heterogeneous data requires selecting minibatches that balance convergence speed with coverage across domains. Existing methods either sele…