16 citations · 53 across the 14 of their papers we have counts for
14 papers
SuperSFL: Resource-Heterogeneous Federated Split Learning with Weight-Sharing Super-Networks
Abdullah Al Asif, Sixing Yu, Juan Pablo Munoz +2
SplitFed Learning (SFL) combines federated learning and split learning to enable collaborative training across distributed edge devices; however, it faces significant challenges in…
PerfMamba: Performance Analysis and Pruning of Selective State Space Models
Abdullah Al Asif, Mobina Kashaniyan, Sixing Yu +2
Recent advances in sequence modeling have introduced selective SSMs as promising alternatives to Transformer architectures, offering theoretical computational efficiency and sequen…
SuperSAM: Crafting a SAM Supernetwork via Structured Pruning and Unstructured Parameter Prioritization
Waqwoya Abebe, Sadegh Jafari, Sixing Yu +6
Neural Architecture Search (NAS) is a powerful approach of automating the design of efficient neural architectures. In contrast to traditional NAS methods, recently proposed one-sh…
PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
Branden Butler, Sixing Yu, Arya Mazaheri +1
Inference of Large Language Models (LLMs) across computer clusters has become a focal point of research in recent times, with many acceleration techniques taking inspiration from C…
The Landscape and Challenges of HPC Research and LLMs
Le Chen, Nesreen K. Ahmed, Akash Dutta +14
Recently, language models (LMs), especially large language models (LLMs), have revolutionized the field of deep learning. Both encoder-decoder models and prompt-based techniques ha…
Bridging the Gap Between Foundation Models and Heterogeneous Federated Learning
Sixing Yu, J. Pablo Muñoz, Ali Jannesari
Federated learning (FL) offers privacy-preserving decentralized machine learning, optimizing models at edge clients without sharing private data. Simultaneously, foundation models…
Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models
Sixing Yu, J. Pablo Muñoz, Ali Jannesari
Foundation Models (FMs), such as LLaMA, BERT, GPT, ViT, and CLIP, have demonstrated remarkable success in a wide range of applications, driven by their ability to leverage vast amo…
Towards Seamless Management of AI Models in High-Performance Computing
Sixing Yu, Murali Emani, Chunhua Liao +4
With the increasing prevalence of artificial intelligence (AI) in diverse science/engineering communities, AI models emerge on an unprecedented scale among various domains. However…
Resource-Aware Heterogeneous Federated Learning using Neural Architecture Search
Sixing Yu, J. Pablo Muñoz, Ali Jannesari
Federated Learning (FL) is extensively used to train AI/ML models in distributed and privacy-preserving settings. Participant edge devices in FL systems typically contain non-indep…
Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusion
Duy Phuong Nguyen, Sixing Yu, J. Pablo Muñoz +1
Concerned with user data privacy, this paper presents a new federated learning (FL) method that trains machine learning models on edge devices without accessing sensitive data. Tra…
SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning
Sixing Yu, Phuong Nguyen, Waqwoya Abebe +3
Federated learning~(FL) facilitates the training and deploying AI models on edge devices. Preserving user data privacy in FL introduces several challenges, including expensive comm…
Heterogeneous Federated Learning using Dynamic Model Pruning and Adaptive Gradient
Sixing Yu, Phuong Nguyen, Ali Anwar +1
Federated Learning (FL) has emerged as a new paradigm for training machine learning models distributively without sacrificing data security and privacy. Learning models on edge dev…
Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning
Sixing Yu, Arya Mazaheri, Ali Jannesari
Model compression is an essential technique for deploying deep neural networks (DNNs) on power and memory-constrained resources. However, existing model-compression methods often r…
Auto Graph Encoder-Decoder for Neural Network Pruning
Sixing Yu, Arya Mazaheri, Ali Jannesari
Model compression aims to deploy deep neural networks (DNN) on mobile devices with limited computing and storage resources. However, most of the existing model compression methods…