Clip Body and Tail Separately: High Probability Guarantees for DPSGD with Heavy Tails

Haichao Sha, Yang Cao, Yong Liu +3

Differentially Private Stochastic Gradient Descent (DPSGD) is widely utilized to preserve training data privacy in deep learning, which first clips the gradients to a predefined no…