137 citations · 233 across the 26 of their papers we have counts for
8 papers · 1 filter
Clean: Second-order LLM Training at Linear Memory Cost via Nystr{ö}m Sketching
Beheshteh T. Rakhshan, Sahar Rajabi, Maziar Sargordi Shikai Fang +4
Training large language models (LLMs) entails a fundamental trade-off: memory-efficient optimizers such as Adam discard cross-parameter curvature, whereas full-curvature methods su…
Geometrically Principled Randomized Optimization for Efficient LLM Training
Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla
Low-rank gradient optimization for large language models is currently divided into two categories: structured methods that rigorously identify subspaces, and randomized approaches…
SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning
Saad Hossain, Samanvay Vajpayee, Sirisha Rambhatla
As large language models (LLMs) become ubiquitous, parameter-efficient fine-tuning methods and safety-first defenses have proliferated rapidly. However, the number of approaches an…
SubTrack++ : Gradient Subspace Tracking for Scalable LLM Training
Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla
Training large language models (LLMs) is highly resource-intensive due to their massive number of parameters and the overhead of optimizer states. While recent work has aimed to re…
Domain Generalization for Domain-Linked Classes
Kimathi Kaai, Saad Hossain, Sirisha Rambhatla
Domain generalization (DG) focuses on transferring domain-invariant knowledge from multiple source domains (available at train time) to an, a priori, unseen target domain(s). This…
Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling
Chuizheng Meng, Sirisha Rambhatla, Yan Liu
Vast amount of data generated from networks of sensors, wearables, and the Internet of Things (IoT) devices underscores the need for advanced modeling techniques that leverage the…
Towards Accurate Spatiotemporal COVID-19 Risk Scores using High Resolution Real-World Mobility Data
Sirisha Rambhatla, Sepanta Zeighami, Kameron Shahabi +2
As countries look towards re-opening of economic activities amidst the ongoing COVID-19 pandemic, ensuring public health has been challenging. While contact tracing only aims to tr…
Physics-aware Spatiotemporal Modules with Auxiliary Tasks for Meta-Learning
Sungyong Seo, Chuizheng Meng, Sirisha Rambhatla +1
Modeling the dynamics of real-world physical systems is critical for spatiotemporal prediction tasks, but challenging when data is limited. The scarcity of real-world data and the…