7 papers
HeLoCo: Efficient asynchronous low-communication training under data and device heterogeneity
Abdullah Al Asif, Patrick Diem, Juan Pablo Muñoz +3
Distributed Low-Communication (DiLoCo) training reduces communication overhead by allowing workers to perform multiple local optimization steps before sending pseudo-gradients to a…
SuperSFL: Resource-Heterogeneous Federated Split Learning with Weight-Sharing Super-Networks
Abdullah Al Asif, Sixing Yu, Juan Pablo Munoz +2
SplitFed Learning (SFL) combines federated learning and split learning to enable collaborative training across distributed edge devices; however, it faces significant challenges in…
CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning
Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz +2
Large language models (LLMs) can acquire new capabilities through fine-tuning, but continual adaptation often leads to catastrophic forgetting. We propose CRAFT, a continual learni…
FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation
Fatema Siddika, Md Anwar Hossen, J. Pablo Muñoz +3
Parameter-efficient fine-tuning (PEFT) adapts large pre-trained models by updating only a small subset of parameters. Recently, Representation Fine-Tuning (ReFT) has emerged as an…
Dual-Distilled Heterogeneous Federated Learning with Adaptive Margins for Trainable Global Prototypes
Fatema Siddika, Md Anwar Hossen, Wensheng Zhang +3
Heterogeneous Federated Learning (HFL) has gained significant attention for its capacity to handle both model and data heterogeneity across clients. Prototype-based HFL methods eme…
PerfMamba: Performance Analysis and Pruning of Selective State Space Models
Abdullah Al Asif, Mobina Kashaniyan, Sixing Yu +2
Recent advances in sequence modeling have introduced selective SSMs as promising alternatives to Transformer architectures, offering theoretical computational efficiency and sequen…