6 citations · 23 across the 13 of their papers we have counts for
13 papers
HeLoCo: Efficient asynchronous low-communication training under data and device heterogeneity
Abdullah Al Asif, Patrick Diem, Juan Pablo Muñoz +3
Distributed Low-Communication (DiLoCo) training reduces communication overhead by allowing workers to perform multiple local optimization steps before sending pseudo-gradients to a…
CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning
Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz +2
Large language models (LLMs) can learn new tasks through fine-tuning, but learning tasks sequentially often leads to catastrophic forgetting of earlier ones. This paper proposes CR…
SuperSFL: Resource-Heterogeneous Federated Split Learning with Weight-Sharing Super-Networks
Abdullah Al Asif, Sixing Yu, Juan Pablo Munoz +2
SplitFed Learning (SFL) combines federated learning and split learning to enable collaborative training across distributed edge devices; however, it faces significant challenges in…
PerfMamba: Performance Analysis and Pruning of Selective State Space Models
Abdullah Al Asif, Mobina Kashaniyan, Sixing Yu +2
Recent advances in sequence modeling have introduced selective SSMs as promising alternatives to Transformer architectures, offering theoretical computational efficiency and sequen…
FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation
Fatema Siddika, Md Anwar Hossen, J. Pablo Muñoz +3
Parameter-efficient fine-tuning (PEFT) adapts large pre-trained models by updating only a small subset of parameters. Recently, Representation Fine-Tuning (ReFT) has emerged as an…
Dual-Distilled Heterogeneous Federated Learning with Adaptive Margins for Trainable Global Prototypes
Fatema Siddika, Md Anwar Hossen, Wensheng Zhang +3
Heterogeneous Federated Learning (HFL) has gained significant attention for its capacity to handle both model and data heterogeneity across clients. Prototype-based HFL methods eme…
Federated Multimodal Learning with Dual Adapters and Selective Pruning for Communication and Computational Efficiency
Duy Phuong Nguyen, J. Pablo Munoz, Tanya Roosta +1
Federated Learning (FL) enables collaborative learning across distributed clients while preserving data privacy. However, FL faces significant challenges when dealing with heteroge…
FLoRA: Enhancing Vision-Language Models with Parameter-Efficient Federated Learning
Duy Phuong Nguyen, J. Pablo Munoz, Ali Jannesari
In the rapidly evolving field of artificial intelligence, multimodal models, e.g., integrating vision and language into visual-language models (VLMs), have become pivotal for many…
The Landscape and Challenges of HPC Research and LLMs
Le Chen, Nesreen K. Ahmed, Akash Dutta +14
Recently, language models (LMs), especially large language models (LLMs), have revolutionized the field of deep learning. Both encoder-decoder models and prompt-based techniques ha…
LEFL: Low Entropy Client Sampling in Federated Learning
Waqwoya Abebe, Pablo Munoz, Ali Jannesari
Federated learning (FL) is a machine learning paradigm where multiple clients collaborate to optimize a single global model using their private data. The global model is maintained…
Bridging the Gap Between Foundation Models and Heterogeneous Federated Learning
Sixing Yu, J. Pablo Muñoz, Ali Jannesari
Federated learning (FL) offers privacy-preserving decentralized machine learning, optimizing models at edge clients without sharing private data. Simultaneously, foundation models…
Resource-Aware Heterogeneous Federated Learning using Neural Architecture Search
Sixing Yu, J. Pablo Muñoz, Ali Jannesari
Federated Learning (FL) is extensively used to train AI/ML models in distributed and privacy-preserving settings. Participant edge devices in FL systems typically contain non-indep…
Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusion
Duy Phuong Nguyen, Sixing Yu, J. Pablo Muñoz +1
Concerned with user data privacy, this paper presents a new federated learning (FL) method that trains machine learning models on edge devices without accessing sensitive data. Tra…