7 papers
CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Models
Huiwen Wu, Xiaogang Xu, Deyi Zhang +3
The success of current Large-Language Models (LLMs) hinges on extensive training data that is collected and stored centrally, called Centralized Learning (CL). However, such a coll…
Consistent Estimation of Numerical Distributions under Local Differential Privacy by Wavelet Expansion
Puning Zhao, Zhikun Zhang, Bo Sun +4
Distribution estimation under local differential privacy (LDP) is a fundamental and challenging task. Significant progresses have been made on categorical data. However, due to dif…
Contextual Bandits for Unbounded Context Distributions
Puning Zhao, Rongfei Fan, Shaowei Wang +4
Nonparametric contextual bandit is an important model of sequential decision making problems. Under -Tsybakov margin condition, existing research has established a regret bound…
A Huber Loss Minimization Approach to Mean Estimation under User-level Differential Privacy
Puning Zhao, Lifeng Lai, Li Shen +3
Privacy protection of users' entire contribution of samples is important in distributed systems. The most effective approach is the two-stage scheme, which finds a small interval f…
Differential Private Stochastic Optimization with Heavy-tailed Data: Towards Optimal Rates
Puning Zhao, Jiafei Wu, Zhe Liu +3
We study convex optimization problems under differential privacy (DP). With heavy-tailed gradients, existing works achieve suboptimal rates. The main obstacle is that existing grad…
Learning with User-Level Local Differential Privacy
Puning Zhao, Li Shen, Rongfei Fan +4
User-level privacy is important in distributed systems. Previous research primarily focuses on the central model, while the local models have received much less attention. Under th…