6 papers
Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning
Fatema Siddika, Md Anwar Hossen, Tanwi Mallick +1
Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previ…
CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning
Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz +2
Large language models (LLMs) can acquire new capabilities through fine-tuning, but continual adaptation often leads to catastrophic forgetting. We propose CRAFT, a continual learni…
Split-on-Share: Mixture of Sparse Experts for Task-Agnostic Continual Learning
Fatema Siddika, Md Anwar Hossen, Tanwi Mallick +1
Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previ…
SCOPE: Semantic Coreset with Orthogonal Projection Embeddings for Federated learning
Md Anwar Hossen, Nathan R. Tallent, Luanzheng Guo +1
Scientific discovery increasingly requires learning on federated datasets, fed by streams from high-resolution instruments, that have extreme class imbalance. Current ML approaches…
FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation
Fatema Siddika, Md Anwar Hossen, J. Pablo Muñoz +3
Parameter-efficient fine-tuning (PEFT) adapts large pre-trained models by updating only a small subset of parameters. Recently, Representation Fine-Tuning (ReFT) has emerged as an…
Dual-Distilled Heterogeneous Federated Learning with Adaptive Margins for Trainable Global Prototypes
Fatema Siddika, Md Anwar Hossen, Wensheng Zhang +3
Heterogeneous Federated Learning (HFL) has gained significant attention for its capacity to handle both model and data heterogeneity across clients. Prototype-based HFL methods eme…