Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level

Saleh Vatan Khah, Savelii Chezhegov, Shahrokh Farahmand +2

Gradient clipping is a fundamental tool in Deep Learning, improving the high-probability convergence of stochastic first-order methods like SGD, AdaGrad, and Adam under heavy-taile…