5 papers
Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models
Lele Zheng, Weifeng Kong, Xinyi Zhang +3
Differentially private zeroth-order optimization (DP-ZO) enables memory-efficient private fine-tuning of large language models using only forward evaluations. Existing aggregation-…
FedGSA: Geometry-Consistent Subspace Aggregation for Differentially Private Federated LoRA
Lele Zheng, Ruijie Hu, Tao Zhang +2
Low-Rank Adaptation (LoRA) enables communication-efficient federated fine-tuning of pretrained language models. However, integrating differential privacy (DP) into federated LoRA r…
When Convenience Becomes Risk: A Semantic View of Under-Specification in Host-Acting Agents
Di Lu, Yongzhi Liao, Xutong Mu +5
Host-acting agents promise a convenient interaction model in which users specify goals and the system determines how to realize them. We argue that this convenience introduces a di…
Differentially Private Subspace Fine-Tuning for Large Language Models
Lele Zheng, Xiang Wang, Tao Zhang +3
Fine-tuning large language models on downstream tasks is crucial for realizing their cross-domain potential but often relies on sensitive data, raising privacy concerns. Differenti…
Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions
Hao Du, Shang Liu, Lele Zheng +3
Fine-tuning has emerged as a critical process in leveraging Large Language Models (LLMs) for specific downstream tasks, enabling these models to achieve state-of-the-art performanc…