4 papers · 1 filter
Unveiling the Depth-Performance Dilemma in Split-Federated Fine-tuning of LLMs
Hariharan Ramesh, Someshwaran Murugaiyan, Jyotikrishna Dass
Split Federated Fine-tuning (SFF) is a promising paradigm for scaling Large Language Models (LLMs) by partitioning model depth between resource-constrained clients and a centralize…
Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers
Hariharan Ramesh, Jyotikrishna Dass
Fine-tuning Vision Transformers (ViTs) with low-rank adapters (LoRA) promises better communication efficiency under federated setup, yet existing aggregation strategies face fundam…
FLoRIST: Singular Value Thresholding for Efficient and Accurate Federated Fine-Tuning of Large Language Models
Hariharan Ramesh, Jyotikrishna Dass
Integrating Low-Rank Adaptation (LoRA) into federated learning offers a promising solution for parameter-efficient fine-tuning of Large Language Models (LLMs) without sharing local…
NetDistiller: Empowering Tiny Deep Learning via In-Situ Distillation
Shunyao Zhang, Yonggan Fu, Shang Wu +4
Boosting the task accuracy of tiny neural networks (TNNs) has become a fundamental challenge for enabling the deployments of TNNs on edge devices which are constrained by strict li…