3 papers
cs.LG2026
Geometrically Principled Randomized Optimization for Efficient LLM Training
Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla
Low-rank gradient optimization for large language models is currently divided into two categories: structured methods that rigorously identify subspaces, and randomized approaches…
cs.LG2025
SubTrack++ : Gradient Subspace Tracking for Scalable LLM Training
Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla
Training large language models (LLMs) is highly resource-intensive due to their massive number of parameters and the overhead of optimizer states. While recent work has aimed to re…
cs.CL2025
GenKnowSub: Improving Modularity and Reusability of LLMs through General Knowledge Subtraction
Mohammadtaha Bagherifard, Sahar Rajabi, Ali Edalat +1
Large language models often struggle with zero-shot generalization, and several modular approaches have been proposed to address this challenge. Yet, we hypothesize that a key limi…