68 citations · 105 across the 19 of their papers we have counts for
8 papers · 1 filter
LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
Jiawei Hao, Zhiwei Hao, Jianyuan Guo +4
Mixture-of-Experts (MoE) based Large Language Models (LLMs) have demonstrated impressive performance and computational efficiency. However, their deployment is often constrained by…
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Anke Tang, Xianglin Luo, Li Shen +5
Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct…
Federated Learning with Manifold Regularization and Normalized Update Reaggregation
Xuming An, Li Shen, Han Hu +1
Federated Learning (FL) is an emerging collaborative machine learning framework where multiple clients train the global model without sharing their own datasets. In FL, the model i…
Parameter Efficient Multi-task Model Fusion with Partial Linearization
Anke Tang, Li Shen, Yong Luo +5
Large pre-trained models have enabled significant advances in machine learning and served as foundation components. Model fusion methods, such as task arithmetic, have been proven…
Improving Heterogeneous Model Reuse by Density Estimation
Anke Tang, Yong Luo, Han Hu +5
This paper studies multiparty learning, aiming to learn a model using the private data of different participants. Model reuse is a promising solution for multiparty learning, assum…
FedABC: Targeting Fair Competition in Personalized Federated Learning
Dui Wang, Li Shen, Yong Luo +4
Federated learning aims to collaboratively train models without accessing their client's local private data. The data may be Non-IID for different clients and thus resulting in poo…
Multi-Agent Collaborative Inference via DNN Decoupling: Intermediate Feature Compression and Edge Learning
Zhiwei Hao, Guanyu Xu, Yong Luo +3
Recently, deploying deep neural network (DNN) models via collaborative inference, which splits a pre-trained model into two parts and executes them on user equipment (UE) and edge…
CLNode: Curriculum Learning for Node Classification
Xiaowen Wei, Xiuwen Gong, Yibing Zhan +3
Node classification is a fundamental graph-based task that aims to predict the classes of unlabeled nodes, for which Graph Neural Networks (GNNs) are the state-of-the-art methods.…