4 papers
FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training
Dikshant Kukreja, Kritarth Prasad, Avinash Anand +6
Reverse-mode differentiation computes every weight gradient, writes it to memory, and only then lets the optimizer read it back. This two-phase schedule sets the memory ceiling of…
Painless Federated Learning: An Interplay of Line-Search and Extrapolation
Geetika, Somya Tyagi, Bapi Chatterjee
The classical line search for learning rate (LR) tuning in the stochastic gradient descent (SGD) algorithm can tame the convergence slowdown due to data-sampling noise. In a federa…
Federated Instrumental Variable Analysis via Federated Generalized Method of Moments
Geetika, Somya Tyagi, Bapi Chatterjee
Instrumental variables (IV) analysis is an important applied tool for areas such as healthcare and consumer economics. For IV analysis in high-dimensional settings, the Generalized…
FedOptimus: Optimizing Vertical Federated Learning for Scalability and Efficiency
Nikita Shrivastava, Drishya Uniyal, Bapi Chatterjee
Federated learning (FL) is a collaborative machine learning paradigm which ensures data privacy by training models across distributed datasets without centralizing sensitive inform…