6 papers · 1 filter
Effective MoE-based LLM Compression by Exploiting Heterogeneous Inter-Group Experts Routing Frequency and Information Density
Zhendong Mi, Yixiao Chen, Pu Zhao +4
Mixture-of-Experts (MoE) based Large Language Models (LLMs) have achieved superior performance, yet the massive memory overhead caused by storing multiple expert networks severely…
Pruning and Malicious Injection: A Retraining-Free Backdoor Attack on Transformer Models
Taibiao Zhao, Mingxuan Sun, Hao Wang +2
Transformer models have demonstrated exceptional performance and have become indispensable in computer vision (CV) and natural language processing (NLP) tasks. However, recent stud…
Towards Interpretable Adversarial Examples via Sparse Adversarial Attack
Fudong Lin, Jiadong Lou, Hao Wang +2
Sparse attacks are to optimize the magnitude of adversarial perturbations for fooling deep neural networks (DNNs) involving only a few perturbed pixels (i.e., under the l0 constrai…
Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
Yide Ran, Wentao Guo, Jingwei Sun +7
Federated Learning enables collaborative fine-tuning of Large Language Models (LLMs) across decentralized Non-Independent and Identically Distributed (Non-IID) clients, but such mo…
An Empirical Study of the Impact of Federated Learning on Machine Learning Model Accuracy
Haotian Yang, Zhuoran Wang, Benson Chou +4
Federated Learning (FL) enables distributed ML model training on private user data at the global scale. Despite the potential of FL demonstrated in many domains, an in-depth view o…
Learning Unlabeled Clients Divergence for Federated Semi-Supervised Learning via Anchor Model Aggregation
Marawan Elbatel, Hualiang Wang, Jixiang Chen +2
Federated semi-supervised learning (FedSemi) refers to scenarios where there may be clients with fully labeled data, clients with partially labeled, and even fully unlabeled client…