Efficient DP-SGD for LLMs with Randomized Clipping

Enayat Ullah, Sai Aparna Aketi, Devansh Gupta +2

Large language models (LLMs) are trained on vast datasets that may contain sensitive information. Differential privacy (DP), the de facto standard for formal privacy guarantees, pr…