Large Language Models Can Be Strong Differentially Private Learners

Abstract

Differentially Private (DP) learning has seen limited success for building large deep learning models of text, and straightforward attempts at applying Differentially Private Stochastic Gradient Descent (DP-SGD) to NLP tasks have resulted in large performance drops and high computational overhead. We show that this performance drop can be mitigated with (1) the use of large pretrained language models; (2) non-standard hyperparameters that suit DP optimization; and (3) fine-tuning objectives which are aligned with the pretraining procedure. With the above, we obtain NLP models that outperform state-of-the-art DP-trained models under the same privacy budget and strong non-private baselines -- by directly fine-tuning pretrained models with DP optimization on moderately-sized corpora. To address the computational challenge of running DP-SGD with large Transformers, we propose a memory saving technique that allows clipping in DP-SGD to run without instantiating per-example gradients for any linear layer in the model. The technique enables privately training Transformers with almost the same memory cost as non-private training at a modest run-time overhead. Contrary to conventional wisdom that DP optimization fails at learning high-dimensional models (due to noise that scales with dimension) empirical results reveal that private learning with pretrained language models doesn't tend to suffer from dimension-dependent performance degradation. Code to reproduce results can be found at https://github.com/lxuechen/private-transformers.

arXiv comment31 pages; update ethics statement to clarify benefits and potential long-term harms

References in corpus (33)

• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Language Models are Few-Shot Learners
• LoRA: Low-Rank Adaptation of Large Language Models
• On the Opportunities and Risks of Foundation Models
• Scaling Laws for Neural Language Models
• Renyi Differential Privacy
• The Curious Case of Neural Text Degeneration
• Learning Differentially Private Recurrent Language Models
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Scalable Private Learning with PATE
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Differentially Private Synthetic Medical Data Generation using Convolutional GANs
• Language Models as Knowledge Bases?
• DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
• Gaussian Differential Privacy
• Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
• Finetuned Language Models Are Zero-Shot Learners
• AdaCliP: Adaptive Clipping for Private SGD
• Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
• The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
• Efficient Per-Example Gradient Computations
• GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators
• Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning
• Opacus: User-Friendly Differential Privacy Library in PyTorch
• Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorization
• Training Production Language Models without Memorizing User Data
• Private Adaptive Gradient Methods for Convex Optimization
• Private Post-GAN Boosting
• On a Utilitarian Approach to Privacy Preserving Text Generation
• Wide Network Learning with Differential Privacy
• Fast Dimension Independent Private AdaGrad on Publicly Estimated Subspaces
• An Efficient DP-SGD Mechanism for Large Scale NLP Models

Cited by in corpus (13)

• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
• How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
• Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey
• DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass
• Differentially Private Fine-tuning of Language Models
• The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare
• A Customized Text Sanitization Mechanism with Differential Privacy
• Large Scale Private Learning via Low-rank Reparametrization
• LazyDP: Co-Designing Algorithm-Software for Scalable Training of Differentially Private Recommendation Models
• Public Data-Assisted Mirror Descent for Private Model Training
• Modelling Direct Messaging Networks with Multiple Recipients for Cyber Deception
• Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models
• Privacy-Aware Document Visual Question Answering