7 citations · 7 across the 11 of their papers we have counts for
10 papers · 1 filter
Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs
Li Shen, Xiaolei Hao, Qinglun Li +3
One-Shot Federated Learning, where a central server learns a global model in a single communication round, has emerged as a promising paradigm. However, under extremely non-IID set…
Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs
Yifei Liang, Yan Sun, Xiaochun Cao +1
Push-Sum-based decentralized learning enables optimization over directed communication networks, where information exchange may be asymmetric. While convergence properties of such…
Sparse Layer Sharpness-Aware Minimization for Efficient Fine-Tuning
Yifei Cheng, Xianglin Yang, Guoxia Wang +5
Sharpness-aware minimization (SAM) seeks the minima with a flat loss landscape to improve the generalization performance in machine learning tasks, including fine-tuning. However,…
Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training
Qinglun Li, Yingqi Liu, Miao Zhang +3
Decentralized training removes the centralized server, making it a communication-efficient approach that can significantly improve training efficiency, but it often suffers from de…
TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models
Yang Dai, Oubo Ma, Longfei Zhang +6
Recent advances in Trajectory Optimization (TO) models have achieved remarkable success in offline reinforcement learning. However, their vulnerabilities against backdoor attacks a…
LightSAM: Parameter-Agnostic Sharpness-Aware Minimization
Yifei Cheng, Li Shen, Hao Sun +3
Sharpness-Aware Minimization (SAM) optimizer enhances the generalization ability of the machine learning model by exploring the flat minima landscape through weight perturbations.…
Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent
Yongxian Wei, Anke Tang, Li Shen +3
Merging multiple expert models offers a promising approach for performing multi-task learning without accessing their original data. Existing methods attempt to alleviate task conf…
Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging
Li Shen, Anke Tang, Enneng Yang +6
Multi-task learning (MTL) leverages a shared model to accomplish multiple tasks and facilitate knowledge transfer. Recent research on task arithmetic-based MTL demonstrates that me…
Boosting the Performance of Decentralized Federated Learning via Catalyst Acceleration
Qinglun Li, Miao Zhang, Yingqi Liu +3
Decentralized Federated Learning has emerged as an alternative to centralized architectures due to its faster training, privacy preservation, and reduced communication overhead. In…
Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities
Enneng Yang, Li Shen, Guibing Guo +4
Model merging is an efficient empowerment technique in the machine learning community that does not require the collection of raw training data and does not require expensive compu…