5 papers
Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models
Lele Zheng, Weifeng Kong, Xinyi Zhang +3
Differentially private zeroth-order optimization (DP-ZO) enables memory-efficient private fine-tuning of large language models using only forward evaluations. Existing aggregation-…
FedGSA: Geometry-Consistent Subspace Aggregation for Differentially Private Federated LoRA
Lele Zheng, Ruijie Hu, Tao Zhang +2
Low-Rank Adaptation (LoRA) enables communication-efficient federated fine-tuning of pretrained language models. However, integrating differential privacy (DP) into federated LoRA r…
Differentially Private Subspace Fine-Tuning for Large Language Models
Lele Zheng, Xiang Wang, Tao Zhang +3
Fine-tuning large language models on downstream tasks is crucial for realizing their cross-domain potential but often relies on sensitive data, raising privacy concerns. Differenti…
Guard-GBDT: Efficient Privacy-Preserving Approximated GBDT Training on Vertical Dataset
Anxiao Song, Shujie Cui, Jianli Bai +3
In light of increasing privacy concerns and stringent legal regulations, using secure multiparty computation (MPC) to enable collaborative GBDT model training among multiple data o…
PriFFT: Privacy-preserving Federated Fine-tuning of Large Language Models via Hybrid Secret Sharing
Zhichao You, Xuewen Dong, Ke Cheng +5
Fine-tuning large language models (LLMs) raises privacy concerns due to the risk of exposing sensitive training data. Federated learning (FL) mitigates this risk by keeping trainin…