5 papers
Democratizing LLM Training with the Edge: Challenges, Systems, and the Road Ahead
Ziyue Luo, Jiaxuan Cai, Cedric LeDenmat +27
The growing computational power of modern GPU data centers has let a few industrial giants train and deploy large language models (LLMs) on massive amounts of data. Yet the sheer s…
Beyond Type-checking: Towards Holistic Evaluation of Formal Specification Generation
Srijith Nair, Aditya Vempaty, Jia Liu +1
When generating verifiable code, natural language requirements are mapped to machine checked code using LLMs and agentic workflows. A crucial component of this pipeline is specific…
PMF-CL: Pareto-Minimal-Forgetting Continual Learner for Conflicting Tasks
Srijith Nair, Atilla Eryilmaz, Jia Liu
In the literature, many continual learning (CL) algorithms have been proposed to address the issue of catastrophic forgetting in ML models (i.e., learning new tasks leads to the lo…
FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation
Srijith Nair, Michael Lin, Peizhong Ju +3
Collaborative training methods like Federated Learning (FL) and Split Learning (SL) enable distributed machine learning without sharing raw data. However, FL assumes clients can tr…
EcoLoRA: Communication-Efficient Federated Fine-Tuning of Large Language Models
Han Liu, Ruoyao Wen, Srijith Nair +6
To address data locality and privacy restrictions, Federated Learning (FL) has recently been adopted to fine-tune large language models (LLMs), enabling improved performance on var…