6 papers
CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Models
Huiwen Wu, Xiaogang Xu, Deyi Zhang +3
The success of current Large-Language Models (LLMs) hinges on extensive training data that is collected and stored centrally, called Centralized Learning (CL). However, such a coll…
Contextual Bandits for Unbounded Context Distributions
Puning Zhao, Rongfei Fan, Shaowei Wang +4
Nonparametric contextual bandit is an important model of sequential decision making problems. Under -Tsybakov margin condition, existing research has established a regret bound…
DR-Encoder: Encode Low-rank Gradients with Random Prior for Large Language Models Differentially Privately
Huiwen Wu, Deyi Zhang, Xiaohan Li +3
The emergence of the Large Language Model (LLM) has shown their superiority in a wide range of disciplines, including language understanding and translation, relational logic reaso…
Iter-AHMCL: Alleviate Hallucination for Large Language Model via Iterative Model-level Contrastive Learning
Huiwen Wu, Xiaohan Li, Xiaogang Xu +3
The development of Large Language Models (LLMs) has significantly advanced various AI applications in commercial and scientific research fields, such as scientific literature summa…
Learning with User-Level Local Differential Privacy
Puning Zhao, Li Shen, Rongfei Fan +4
User-level privacy is important in distributed systems. Previous research primarily focuses on the central model, while the local models have received much less attention. Under th…
Enhancing Learning with Label Differential Privacy by Vector Approximation
Puning Zhao, Rongfei Fan, Huiwen Wu +3
Label differential privacy (DP) is a framework that protects the privacy of labels in training datasets, while the feature vectors are public. Existing approaches protect the priva…