3 papers
cs.LG2026
GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection
Kai Yao, Zhenghan Song, Kaixin Wu +5
Parameter-Efficient Fine-Tuning (PEFT) has become a key strategy for adapting large language models, with recent advances in sparse tuning reducing overhead by selectively updating…
cs.CL2025
GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models
Kai Yao, Zhaorui Tan, Penglei Gao +7
The rapid growth of large language models (LLMs) with traditional centralized fine-tuning emerges as a key technique for adapting these models to domain-specific challenges, yieldi…
cs.CL2024
ScaleOT: Privacy-utility-scalable Offsite-tuning with Dynamic LayerReplace and Selective Rank Compression
Kai Yao, Zhaorui Tan, Tiandi Ye +5
Offsite-tuning is a privacy-preserving method for tuning large language models (LLMs) by sharing a lossy compressed emulator from the LLM owners with data owners for downstream tas…