Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Geometrically Principled Randomized Optimization for Efficient LLM Training
Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla
Low-rank gradient optimization for large language models is currently divided into two categories: structured methods that rigorously identify subspaces, and randomized approaches…
cs.LG2025
SubTrack++ : Gradient Subspace Tracking for Scalable LLM Training
Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla
Training large language models (LLMs) is highly resource-intensive due to their massive number of parameters and the overhead of optimizer states. While recent work has aimed to re…