4 papers
GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection
Kai Yao, Zhenghan Song, Kaixin Wu +5
Parameter-Efficient Fine-Tuning (PEFT) has become a key strategy for adapting large language models, with recent advances in sparse tuning reducing overhead by selectively updating…
Towards Instance-wise Personalized Federated Learning via Semi-Implicit Bayesian Prompt Tuning
Tiandi Ye, Wenyan Liu, Kai Yao +6
Federated learning (FL) is a privacy-preserving machine learning paradigm that enables collaborative model training across multiple distributed clients without disclosing their raw…
GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models
Kai Yao, Zhaorui Tan, Penglei Gao +7
The rapid growth of large language models (LLMs) with traditional centralized fine-tuning emerges as a key technique for adapting these models to domain-specific challenges, yieldi…
ScaleOT: Privacy-utility-scalable Offsite-tuning with Dynamic LayerReplace and Selective Rank Compression
Kai Yao, Zhaorui Tan, Tiandi Ye +5
Offsite-tuning is a privacy-preserving method for tuning large language models (LLMs) by sharing a lossy compressed emulator from the LLM owners with data owners for downstream tas…