Large Language Models Can Be Strong Differentially Private Learners

Abstract

Differentially Private (DP) learning has seen limited success for building large deep learning models of text, and straightforward attempts at applying Differentially Private Stochastic Gradient Descent (DP-SGD) to NLP tasks have resulted in large performance drops and high computational overhead. We show that this performance drop can be mitigated with (1) the use of large pretrained language models; (2) non-standard hyperparameters that suit DP optimization; and (3) fine-tuning objectives which are aligned with the pretraining procedure. With the above, we obtain NLP models that outperform state-of-the-art DP-trained models under the same privacy budget and strong non-private baselines -- by directly fine-tuning pretrained models with DP optimization on moderately-sized corpora. To address the computational challenge of running DP-SGD with large Transformers, we propose a memory saving technique that allows clipping in DP-SGD to run without instantiating per-example gradients for any linear layer in the model. The technique enables privately training Transformers with almost the same memory cost as non-private training at a modest run-time overhead. Contrary to conventional wisdom that DP optimization fails at learning high-dimensional models (due to noise that scales with dimension) empirical results reveal that private learning with pretrained language models doesn't tend to suffer from dimension-dependent performance degradation. Code to reproduce results can be found at https://github.com/lxuechen/private-transformers.

31 pages; update ethics statement to clarify benefits and potential long-term harms

References in corpus (12)

• Language Models are Few-Shot Learners
• LoRA: Low-Rank Adaptation of Large Language Models
• Scaling Laws for Neural Language Models
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Gaussian Differential Privacy
• Finetuned Language Models Are Zero-Shot Learners
• Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
• The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
• Opacus: User-Friendly Differential Privacy Library in PyTorch
• Training Production Language Models without Memorizing User Data
• Private Adaptive Gradient Methods for Convex Optimization
• On a Utilitarian Approach to Privacy Preserving Text Generation

Cited by in corpus (13)

• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
• How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
• Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey
• DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass
• Differentially Private Fine-tuning of Language Models
• The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare
• A Customized Text Sanitization Mechanism with Differential Privacy
• Large Scale Private Learning via Low-rank Reparametrization
• LazyDP: Co-Designing Algorithm-Software for Scalable Training of Differentially Private Recommendation Models
• Public Data-Assisted Mirror Descent for Private Model Training
• Modelling Direct Messaging Networks with Multiple Recipients for Cyber Deception
• Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models
• Privacy-Aware Document Visual Question Answering